NATURAL LANGUAGE

PROCESSING (AM622I2A)
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Natural Language Processing (NLP)
• Language is the primary means of communication used by us
▪ Tool used to express our emotions, ideas, information etc.
• NLP is concerned with the development of computational
models of aspects of human language processing.
▪ It is a field of Artificial Intelligence (AI) and Computer Science
that is concerned with the interactions between computers
and humans in natural language.
▪ The goal is to develop algorithms and models that enable
computers to understand, interpret, generate, and manipulate
human languages.

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
• Historically, there were two major approaches to NLP
▪ rationalist approach
• assumes the existence of some language faculty in the
human brain.
▪ empiricist approach
• do not believe in existence of g language faculty.
• believe in the existence of some general, organization
principles such as pattern recognition, generalization,
association etc.

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Origin of NLP
• Sometimes NLP Termed as NLU (Natural language
understanding)
▪ originated from machine translation
• But NLP involves Both NLU and NLG(Natural Language
Understanding & Generation).
• NLP combines computational linguistics—rule-based modeling of
human language—with statistical, machine learning, and deep
learning models.
• •Computational linguistics is concerned with the study of
language using computational models of linguistic phenomena.
▪ It deals with the application of linguistic theories and computational
techniques for NLP

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
• Computational models
▪ Knowledge driven
• rely on explicitly coded linguistic knowledge
• expressed as a set of grammar rules
▪ data-driven
• Rely on existence of a large amount of data
• Uses machine learning techniques to learn syntactic
patterns.

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Language and Knowledge
• Language is the medium of expression in which
knowledge is deciphered
▪ It can be a knowledge representation tool
▪ Represented the whole body of knowledge which can be
modified by generating of new words or including new
ideas and situation
• Language (text) processing has different levels
▪ each involves different types of knowledge.

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Language (text) processing
• lexical analysis, is the analysis of words which is the fundamental for natural
language
▪ Word-level processing requires morphological knowledge, i.e., knowledge about the
structure and formation of words from basic units (morphemes).
• A morpheme is the smallest meaningful unit of grammar with meaning and cannot
be broken down into smaller units.
• The rules for forming words from morphemes are language specific.
• Eg: the word “unbreakable” can be broken down into “un-“, “break”, and “-able”.
• syntactic analysis, which considers a sequence of words as a unit
▪ Decomposes a sentence into words and identifies the relation between them.
▪ It captures grammaticality or non-grammaticality of sentences by looking at
constraints like word order, number, and case agreement.
• This level of processing requires syntactic knowledge,
▪ i.e., knowledge about how words are combined to form larger units such as phrases
and sentences, and what constraints are imposed on them.
▪ Eg: Sentence 1: Rama is a wise man. Sentence 2: Is Rama a man wise?
▪ Sentence 1 is Syntactically correct
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Language (text) processing contd..
• Semantic analysis which is associated with the meaning of the language.
▪ Concerned with creating meaningful representation of linguistic inputs.
• Eg: “Boiling ice cream “ not semantically correct.
• Discourse analysis is a higher level of analysis
▪ It attempts to interpret the structure and meaning of larger units, e.g., at the
paragraph and document level, in terms of words, phrases, clusters, and sentences
▪ Discourse Analysis is extracting the meaning out of the corpus or text.
▪ It deals with how the meaning of a sentence is determined by preceding sentence.
• Pragmatic analysis is the highest level of analysis
▪ deals with the purposeful use of sentences in situations.
▪ It requires knowledge of the world, i.e., knowledge that extends beyond the
contents of the text
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Challenges of NLP
• As the natural languages are highly ambiguous and vague, achieving precise
representation for language computing is more difficult.
• The inability to capture all the required knowledge.
• Identifying the semantics of sentences.
▪ Sentence meaning can be inferred by the syntactic and semantic relation of words.
▪ The frequency of a word being used in a particular sense also affects its meaning.
▪ Idioms, metaphor, and ellipses add more complexity to identify the meaning of the
written text.
• Also words alone donor make a sentence.
Eg : I do not like icecream
Do not I ice cream like -> is wrong
Eg: Kabir and Ayan are married.
Kabir and Suhana are married - Both gives different meaning
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Challenges of NLP contd..
• The scope of quantifiers (the, each, etc.) is often not clear and poses
problem in automatic processing.
• Ambiguity of natural languages
▪ word level ambiguity
• Eg: bank, can
▪ structural ambiguity
• A sentence may be ambiguous even if the words are not.
• Eg: Stolen icecandy found by tree
• Diversity of grammars to describe the structure of sentences.
▪ there are an infinite number of ways to generate them, which makes writing
grammar rules, and grammar itself, extremely complex.
▪ almost impossible for grammar to capture the structure of all and only
meaningful text
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Idioms, metaphors and ellipsis
• An idiom is a phrase that, when taken as a whole, has a meaning you wouldn't be able to
deduce from the meanings of the individual words.
▪ The ball is in your court.
▪ A piece of cake
▪ Every cloud has a silver lining
▪ Not one’s cup of tea
• A metaphor is a figure of speech that describes something by saying it's something else.
▪ Life is a highway.
▪ Her eyes were diamonds.
▪ He is a shining star.
• An ellipsis is a punctuation mark of three dots (. . .) that shows an omission of words,
represents a pause, or suggests there's something left unsaid.
▪ You mean he's...
▪ The chancellor sweated profusely and stammered, “Your highness, … I wasn’t … I didn’t mean
to imply that … I would never question your wisdom.”

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Language and Grammar
• Automatic processing of language requires the rules and
exceptions of a language to be explained to the computer.
• Grammar defines language.
▪ It consists of a set of rules that allows us to parse and
generate sentences in a language.
• The main hurdle in language specification is constantly
changing nature of natural languages and the presence of a
large number of hard to specify exceptions
▪ leads to the development of a number of grammars
• transformational grammar, lexical functional grammar,
government and binding, dependency grammar, Paninian
grammar, and tree-adjoining grammar.
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Transformational grammar by Noam Chomsky
• Transformational grammar, also known as Transformational Generative Grammar
(TGG), is a theory of language developed by Noam Chomsky.
▪ It describes how sentences are structured and processed in the human brain.
• Generative grammar uses a set of rules to specify or generate all and only grammatical (well-
formed) sentences in a language
▪ It recognizes the relationship among the various elements of a sentence and among the
possible sentences of a language and uses rules (some of which are called transformations)
to express these relationships
• Transformational grammar assigns a “deep structure” and a “surface structure” to show
the relationship of such sentences.
▪ Deep structure represents the core semantic relations of a sentence and surface
structure represents the actual spoken or written form of a sentence.
▪ Surface structures are the versions of sentences that are seen or heard, while deep
structures contain the basic units of meaning of a sentence
▪ The mapping from deep structure to surface structure is carried out by
transformations
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Transformational grammar contd..
Surface Structure
• Deep structure can be transformed in a number of S S
ways to yield many different surface-level
representations.
• Sentences with different surface-level
representation, having the same meaning, share a NP VP NP VP
common deep-level representation
• Example sentences:
Pooja plays veena Pooja Plays Veena Veena Is played by Pooja
Veena is played by Pooja S
▪ Above sentences have same meaning but different
surface structures. Subject Object
▪ They have same deep structure Relation
• Deep subject is Pooja Pooja Veena
• Deep object is veena plays
• Eg:John hits Fred (deep structure)
• John hit Fred, John was hitting Fred, Fred was hit by Deep Structure
John (surface structure)
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Transformational grammar contd…
• Transformational grammar is a set of transformation rules, which
transform one phrase-maker into another phrase-marker.
▪ These rules are applied on the terminal string generated by phrase structure
rules
• Transformational grammar has three components
▪ Phrase structure grammar.
▪ Transformational rules
▪ Morphophonemic rules
• These rules match each sentence representation to a string of phonemes.
• Each of these components consists of a set of rules.

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Transformational grammar contd…
• Phrase structure grammar consists of rules that generate natural language
sentences and assign a structural description to them.
▪ These rules define the deep structure of sentences by specifying how words and
phrases are hierarchically arranged.
• Eg:
S→ NP+VP S → Sentence
NP → Noun Phrase
VP → V+NP VP → Verb Phrase
NP → Det + Noun Det → Determinants
Aux->Auxiliary Verb
V → Aux + Verb
Det → the, a, an
Verb → catch, write, eat
Noun → police , snatcher
Aux → will ,is, can
• Sentence that can be generated by these rules are termed as grammatical.
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
• S → Sentence
▪ A sentence consists of a Noun Phrase (NP) and a Verb Phrase (VP).
• Example: "The boy is playing.“
• NP → Noun Phrase
▪ A noun phrase consists of a Determiner (Det) and a Noun (N).
• Example:"The cat" (Det = "The", N = "cat")“
• A dog" (Det = "A", N = "dog")
• VP → Verb Phrase
▪ A verb phrase consists of a Verb (V) and may include a Noun Phrase (NP) or an
Auxiliary Verb (Aux).
• Example:"is sleeping" (Aux = "is", V = "sleeping")
• "ate the apple" (V = "ate", NP = "the apple")
• Det → Determiner
▪ A determiner specifies a noun, such as articles (a, the), demonstratives (this, that), or
possessives (my, his).
• Example:"The book" (Det = "The")
• "A house" (Det = "A")
• Aux → Auxiliary Verb
▪ Auxiliary verbs help the main verb by indicating tense, aspect, mood, etc.
• Example:"He is running." (Aux = "is", V = "running")
• "She has eaten." (Aux = "has", V = "eaten")
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Ex: "Rama is playing football in the ground”
• S → NP VP
▪ NP (Noun Phrase): "Rama“
▪ VP (Verb Phrase): "is playing football in the ground“
• NP → N
▪ "Rama" (Noun only, no determiner)
• VP → Aux V NP PP
▪ Aux → "is“
▪ V → "playing“
▪ NP (Object) → Det N
• "football" (No determiner here)
• PP → P NP
▪ P → "in“
▪ NP → Det N ("the ground")
• Det → "the“
• N → "ground"
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Transformational grammar contd…
• Transformational rules modify the deep structure to create various
surface structures, such as forming passive sentences, questions, or
negations.
▪ They are heterogeneous and have more than one Symbol on their left
hand side.
▪ These rules are used to transform one surface representation into another,
• e.g., an active sentence into passive one.
• The rule relating active and passive sentences is
NP1-Aux-V-NP2 → NP2-Aux+be+en-V- by - NP1
▪ Above rule says N-Aux-V-NP can be transformed to NP-Aux+be –en-V-by+NP
▪ It involves addition of strings ‘be’ and ‘en’ and certain rearrangement of the
constituents of a sentence.
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Transformational grammar contd…
• Active Sentence:
▪ The teacher (NP₁) is (Aux) teaching (V) the students (NP₂).
• Applying the Rule:
▪ The students (NP₂) are (Aux) being (be) taught (en-V) by the teacher (by + NP₁).
• Passive Sentence:
▪ The students are being taught by the teacher.
• Transformational Rules can either be obligatory or Optional.
▪ Obligatory transformation ensures agreement in number of subjects and verbs
▪ Optional transformation modifies the structure of a sentence while preserving
its meaning.

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Examples..
• Active:
▪ The chef is cooking a delicious meal.
• (NP₁ = The chef, Aux = is, V = cooking, NP₂ = a delicious meal)
• Passive:
▪ A delicious meal is being cooked by the chef.
• Transformation:
▪ NP₂ ("a delicious meal") moves to the subject position.
▪ Aux ("is") remains.
▪ "be" + past participle of "cooking" ("being cooked") is inserted.
▪ "by" + NP₁ ("by the chef") is added.
• Active:
▪ The manager will sign the contract.
• (NP₁ = The manager, Aux = will, V = sign, NP₂ = the contract)
• Passive:
▪ The contract will be signed by the manager.

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Transformational grammar contd…
• Morphophonemic rules match each sentence representation to a string of
phonemes.
▪ These rules convert the strings produced by the transformational rules
into actual sounds of sentences.
▪ These rules handle word forms, ensuring correct grammatical structure by
applying necessary changes in morphology and phonetics.
▪ They describe how morphemes (smallest units of meaning) change their
pronunciation (phonemes) based on their phonetic environment.
• Phonetic environment refers to the position and surrounding sounds in
which a particular sound (phoneme) occurs within a word or sentence.
• Example:
▪ Verb transformation: "go" → "went" (past tense), Think → Thought
▪ Plural formation: "child" → "children“, cat → cats, bus → buses

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Transformational grammar contd…
• The passive transformation rules will convert the sentence into:
The + snatcher + will + be + en + catch + by + police
• Another transformational rule will then reorder ‘en + catch’ to ‘catch + en’
• Then, one of the morphophonemic rules will convert ‘catch + en’ to ‘caught’.
S S

NP VP
NP VP

The snatcher Verb NP

The Police Verb NP

Aux V by NP
Aux V Det Noun

will en catch Det

snatcher Noun
will catch the
be
the police
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Example for Transformational grammar
• Sentence : Jane likes the icecream
• Phrase structure rules
▪S → NP + VP
▪ NP1 → N
▪ VP → V + NP
▪ NP2 → aux + N
▪ N1 → Jane
▪V → likes
▪ Det → the
▪ N2 → icecream

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Apply transformational grammar for following sentences

• The dog is chasing the cat → The cat is being chased by

the dog.
• The boy has repaired the bicycle → The bicycle has been
repaired by the boy

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Processing Indian Languages
• Differences between Indian languages and English
▪ Unlike English, Indic scripts have a non-linear structure.
▪ Unlike English, Indian languages have SOV (Subject-Object-Verb) as the
default sentence structure.
▪ Indian languages have a free word order, i.e., words can be moved freely
within a sentence without changing the meaning of the sentence.
▪ Spelling standardization is more subtle in Hindi than in English
▪ Indian languages have a relatively rich set of morphological variants.
▪ Indian languages make extensive and productive use of complex predicates
(CPs).
▪ Indian languages use post-position (Karakas) case markers instead of
prepositions.
▪ Indian languages use verb complexes consisting of sequences of verbs
• Eg: ga raha hai - singing, khel rahi hai - playing

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
NLP Applications
• Machine Translation
▪ automatic translation of text from one human language to another.
• In order to carry out this translation, it is necessary to have an understanding of
words and phrases, grammars of the two language, involved, semantics of the
languages, and world knowledge.
• Speech Recognition
▪ Is the process of mapping acoustic speech signals to a set of words.
▪ Issues might occur due to wide variations in the pronunciation of words homonym
(e.g. dear and deer) and acoustic ambiguities (e.g., in the reg and interest).
• Speech Synthesis
▪ Speech synthesis refers to automatic production of speech utterance of natural
language sentences
• The systems can read out the mails on telephone, or read out a storybook.
▪ In order to generate utterances, text has to be processed.
• Natural Language Interfaces to Databases
▪ Natural language interfaces allow querying a structured database using natural
language sentences.
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
NLP Applications contd..
• Information Retrieval
▪ Identifies the documents relevant to a user’s query.
▪ NLP techniques such as Indexing (stop word elimination, stemming, phrase
extraction, etc.), word sense disambiguation, query modification, and knowledge
bases have also been used in IR system to enhance performance.
• Eg: Google Search
▪ WordNet, LDOCE (Longman Dictionary of Contemporary English) and Roget’s
Thesaurus are some of the useful lexical resources for IR research.
• Information Extraction
▪ Captures and outputs factual information contained within a document.
▪ the information need is specified as pre-defined database schemas or templates.
• extracting summaries from vast collections of text like Wikipedia, conversational
AI systems like chatbots, extracting stock market announcements from financial
news

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
NLP Applications contd..
• Question Answering
▪ Attempts to find the precise answer, or at least the
precise portion of text in which the answer appears.
▪ Requires precise analysis of questions and portions of
texts, semantics and background knowledge to answer-
certain type of questions.
• Text summarization
▪ Creates the summaries of documents and involves
syntactic, semantic, and discourse level processing

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Information Retrieval
• The availability of a large amount of text in electronic form has
made it extremely difficult to get relevant information.
• Information is always associated with some data (text, number,
image etc..
• ‘retrieval’ refers to the operation of accessing information from
some computer-based representation.
• Retrieval of information requires information to be processed and
stored.
• Information retrieval locates the information relevant to the needs
expressed in the form of query.
▪ It is concerned with the organization, storage, retrieval, and
evaluation of information relevant to the query.
• Information retrieval deals with unstructured data.
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Information Retrieval contd..
• The retrieval is performed based on the content of the document rather
than on its structure.
▪ The IR systems usually return a ranked list of documents.
▪ The IR components have been traditionally incorporated into different types of
information systems including database management systems, bibliographic text
retrieval systems, question answering systems, and more recently in search engines.
▪ Current approaches can be broadly classified into two categories.
• The first category consists of approaches that construct topic hierarchy, e.g.,
Yahoo.
▪ This helps the user locate documents of interest manually by traversing the hierarchy.
▪ cost ineffective and inapplicable due to rapid growth of documents on the Web.
• The second category consists of approaches that rank the retrieved documents
according to relevance.

Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Major Issues in Information Retrieval
1. Choosing a proper representation of the document.
▪ Most human knowledge is coded in natural language, which is difficult to use
as knowledge representation language for computer systems
2. Most of the current retrieval models are based on keyword representation.
▪ This representation creates problems during retrieval due to polysemy,
homonymy, and synonymy.
• Polysemy involves the phenomenon of a lexeme with multiple meaning.
▪ Eg: sound, bank
• Homonymy is an ambiguity in which words that appear the same have unrelated
meanings.
▪ Eg: ring, bat
• Synonymy creates problem when a document is indexed with one term and the query
contains a different term, and the two terms share a common meaning.
▪ Eg: good-fine
▪ Keyword based retrieval ignores semantic and contextual information in the
retrieval process.
• This information is lost in the extraction of keywords from the text and cannot be
recovered by the retrieval algorithms.
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management
Major Issues in Information Retrieval contd..
3. inappropriate characterization of queries by the user
▪ Reasons may be lack of knowledge of the subject or the inherent vagueness
of the natural language or not including relevant terms in the query or
including irrelevant terms.
▪ leads to poor retrieval performance.
4. Matching query representation with that of the document
▪ As the number of similarity measures occurs in the literature, Selection of the
appropriate similarity measure is a crucial issue.
5. Evaluating the performance of IR systems
▪ Recall and precision are the most widely used measure of effectiveness.
6. As the major goal of IR is to search for most relevant document, Measuring the
degree of relevance is subjective in nature.
7. The size of document collections and the varying needs of users also complicit
text retrieval.
Dr. Pushpalatha K
Professor
Sahyadri College of Engineering & Management