Natural Language

Processing [NLP]
BAI601
 Module-1: Syllabus
• Introduction: What is Natural Language Processing?
• Origins of NLP,
• Language and Knowledge,
• The Challenges of NLP,
• Language and Grammar,
• Processing Indian Languages,
• NLP Applications.
• Language Modeling: Statistical Language Model - N-gram
model (unigram, bigram),
• Paninion Framework,
• Karaka theory.
What is NLP ?
• NLP is concerned with development of computational models
of aspects of human language processing [Siddiqui, Tiwari].
Objectives:
• To develop automated tools for language processing.
• To gain a better understanding of human communication.
Inputs:
• Knowledge of how human acquire, store and process language.
• Knowledge of the world
• Knowledge of the language
 Historic approaches to NLP
1. Rationalist – assumes existence of some language faculty
(cognitive capacity shared by all normal human beings) in human
brain.

2. Empiricist – Do not believe in the existence of a language

faculty. Assume existence of general organization principles
such as pattern recognition, generalization and association.

Learning of detailed structures take place through application of

these principles on sensory inputs available to child.
Focus
• We will be concerned with various ways in which computer
systems can analyze and interpret texts.
• We will assume for convenience that these texts are presented
in an electronic format.

Why electronic texts??

• Huge amount of text we can access via the World Wide Web.
Increasing availability of electronic versions of newspapers,
novels, textbooks and indeed subject guides.

Spoken Form
• Serves as our primary means of coordinating our day-to-day
behavior with others.
Written Form-Text
• Serves as a long-term record of knowledge from one
generation to the next.

• Words that are written or printed on a flat surface

(paper, card, street signs and so on) OR displayed on a
screen or electronic device in order to be read by their
intended recipient (or by whoever happens to be
passing by).
• NLP is defined as the branch of AI that provides computers
with the capability of understanding text and spoken words in
the same way a human being can.
• It incorporates ML models, statistics, and DL models into
computational linguistics i.e. rule-based modeling of human
language to allow computers to understand text, spoken words
and understands human language, intent, and sentiment.
• Hence NLP gives the machines the ability to read, understand
and derive meaning from human languages.
• NLP combines the field of linguistics and computer science to
decipher language structure and guidelines, and to make
models which can comprehend, break down and separate
significant details from text and speech.
 Origin of NLP
• Within CS, it relates to formal language theory, compiler
techniques, theorem proving, machine learning and human-
computer interaction.
• NLP is essentially multidisciplinary:
• Mathematics/Computational Linguistics (especially logic).
• Cognitive Science
• Linguistics
• Psycholinguistics
• Philosophy

The NLP Communication Model can

help us appreciate how our unconscious
mind filters our experiences through
our personal internal values, beliefs and
memories; so that effectively we all
construct our own reality.
• Linguists - Study the structure of language itself
• Theoretical linguists mainly provide structural description of natural
language and its semantics.
• Psycholinguists - Study the processes of human language production
and comprehension.
• [Philosophers - Consider what it means to have beliefs, goals and
intentions. How these cognitive abilities relate to language]
• [Identify the structure and meaning of words, sentences, texts and
conversations. Deep understanding of broad language.]
• Computational Linguists - Study of language using computational
models of linguistic phenomena.
Computational Models may be classified under 2 categories:
1. Knowledge Driven
• Rely on explicitly coded linguistic knowledge
• Expressed as a set of handcrafted grammar rules.
Problems
• Acquiring and encoding such knowledge is difficult
• Constraints—lack of sufficient coverage of domain knowledge

2. Data-driven
• Assume existence of a large amount of data
• Usually employ some Machine Learning techniques to learn syntactic
patterns.
Advantages
• Less human effort
• Performance of such systems depend on quantity of data
• Such systems are adaptive to noisy data
NLP – usage of term – in practice
• Mistakenly termed as Natural Language Understanding
• Understanding only involves interpretation of language
• NLP includes both Understanding (interpretation) and Generation
(production)
• NLP also includes Speech processing
• NLU is considered AI-complete
Consider the following example:
Sentence A: “Get me something I can drink from”. Sentence B: “Get me a cup”.
The two sentences A and B are different but have the same meaning.
The AI system needs to have a more meaningful model of language in order for
it to conclude that: A = B

• So NLU in NLP is particularly the hardest one to crack.

• Calling a problem AI-complete, reflects an attitude that it would not be solved
by a simple specific algorithm.
 Why NL Understanding is hard?
• Natural language is extremely rich in form and
structure, and very ambiguous.
• How to represent meaning,
• Which structures map to which meaning structures.
• One input can mean many different things.
Ambiguity can be at different levels.
• Many input can mean the same thing.
• Interaction among components of the input is
not clear.
 Knowledge of Language
• Phonology – concerns how words are related to the
sounds that realize them.

• Morphology – concerns how words are constructed from

more basic meaning units called morphemes. A morpheme
is the primitive unit of meaning in a language.

• Syntax – concerns how can be put together to form correct

sentences and determines what structural role each word
plays in the sentence and what phrases are subparts of
other phrases.

• Semantics – concerns what words mean and how these

meaning combine in sentences to form sentence meaning.
The study of context-independent meaning.
 Knowledge of Language (cont.)
• Pragmatics – concerns how sentences are used in
different situations and how use affects the interpretation
of the sentence.

• Discourse – concerns how the immediately preceding

sentences affect the interpretation of the next sentence.
For example, interpreting pronouns and interpreting the
temporal aspects of the information.

• World Knowledge – includes general knowledge about

the world. What each language user must know about the
other’s beliefs and goals.
 Language and Knowledge

• Text and Content

• Computers
• Map the contents
 Lexical Analysis
• This phase involves analysis of words.
• It requires morphological analysis.
• Morphology is the study of the structure and formation of words.
• One of the widespread task here is lemmatizing or stemming
which is used in many web search engines.
• Ex: dog-dogs, run-ran, bus-buses etc.
• Its most important unit is the Morpheme, which is defined as the
"minimal unit of meaning”.
• Consider a word like: "unhappiness". This has three parts:
 Syntactic Analysis
• It involves analysis of words in the sentence for grammar and arranging words
in a manner that shows the relationship among the words.
• This exploit the results of Morphological analysis to build a structural
description of the sentence.
• It uses the knowledge about how words are combined to form larger units such
as phrases and sentences and what constraints are on them.
• Eg: Valid sentence: “I went to the Market”.
Invalid Sentence: “went the I market to”.
• One of the wide spread task involved here is Parts of Speech Tagging and
constructing Syntax Trees.
• It requires detailed knowledge about rules of Grammar.
 Semantic Analysis
• The structures created by the syntactic analyzer are assigned meanings.
• A mapping is made between the syntactic structures and objects in the
task domain.
• Semantic Analysis must do two important things:
• Must map individual terms into appropriate objects in KB
• Must create correct structures to correspond to correct meaning of
words.
• Semantics and its understanding as a study of meaning covers most
complex tasks like:
• Finding synonyms,
• Word sense disambiguation,
• Constructing question-answering systems, etc..
• The sentence "you have a colorless green ideas....” would be rejected
semantically because colorless & green makes no sense.
 Discourse Analysis
• The meaning of an individual sentences may
• depend on the sentences that precede it and that follow it.
• May influence the meanings of the sentence
• Eg.“Bill had a red balloon. John wanted it.”
• “It” refers to red balloon and it should be identified like that only.
• Such references are called Anaphora.
• Another Example:
“My house was broken into last week.
They took the TV and Music system.”
• “They” should be recognized as referring to burglars
• The Attempts to interpret the structure and meaning of units larger than
sentence such as paragraph, document etc. in terms of words, phrases,
clusters and sentences.
• It requires the discourse knowledge, i.e., knowledge of how meaning of
sentence is determined by preceding sentence.
• Example: Radha is a girl. She went to the market. It was too rush
 Pragmatic Analysis
• The structure representing what was said is reinterpreted to
determine that what was actually meant.

• For example, the sentence “Do you know what time it is?” should be
interpreted as a request to be told the time.

• This is the final step towards what to do as a result.

• One way is to record what was said as fact. (for some sentences
intended effect is declarative and sometimes it is not).
 Challenges
• Related to the representation and interpretation like human.
1. Representation of meaning of a sentence, meaning of words
appearing in it.
• Meaning of word and its use in language.
• Example: ‘I like Ice cream’ instead if we use ‘like Ice cream I’

• Words as well as their syntactic and semantic relation that gives

meaning to a sentence.
• Example: ‘Kabir and Ayan are Married’ and ‘Kabir and Suha are
Married’

• Language keeps evolving

• New words are added continually.
• Existing words are introduced in new context.
2. Machine must rely on word contexts to learn the meaning of specific word
in a message.
• Context depends on co-occurring words (occurring before or after a word).

• Word Frequency in a particular sense also affects its meaning.

Example: ‘while’ as conjunction or as ‘a short interval of time’
You can go swimming while I am having food.
Once in a while it happened so.

3. Idioms, metaphor and ellipses add to complexity in identifying meaning

of written text.
• Example: Meaning of ‘The old man finally kicked the bucket’ has nothing to
do with words ‘kick’ and ‘bucket’.

• Idiom - it’s piece of cake- it’s easy

• Metaphor - Time is money- something is referred to something
• Ellipses – were you thinking about me today…?
4. Quantifier scoping (the, each, etc) is not clear and poses problem in
automatic processing.

5. Ambiguity of natural languages is another difficulty.

• Ambiguity at word level: we can identify words that have multiple meanings
associated with them.
• Example: ‘can’ (as verb in ‘can play’/as noun-as ‘Empty can’ means that
container)
‘bank’ (financial institution or as river side)
‘bat’ (mammal or playing equipment)
• Structure Ambiguity-Ambiguity at sentence level: None of the words are
ambiguous, but the sentence is.
• Example: ‘Stolen rifle found by the tree’

6. Incorporating contextual and world knowledge (culture, language,

traditions,…) is greatest difficulty in language computing.
• Example: ‘Taj’ means a monument, a brand of tea, or a hotel to an Indian but
need not to be so for non-Indian.
 Language and Grammar

• Automatic processing of language requires the rules and

exceptions of language to be explained to the computer.
• Grammar defines language
• It consists of a set of rules that allow us to parse and generate
sentences of a language.
• Hurdle in Language Specification
• Constantly changing nature of natural languages
• Presence of a large number of hard-to-specify exceptions.
• Transformational Grammar (Noam Chomsky, 1957)
• Lexical Functional Grammar (Kaplan and Bresnan,
1982) (Relationships)
• Government and binding (Chomsky, 1981)
• Generalized phrase structure grammar (Derivation)
• Dependency grammar (Relationships)
• Paninian grammar (Relationships)
• Tree-adjoining grammar (Joshi, 1985)
 Transformational Grammar –
Chomsky, 1957
• Proposed by Noam Chomsky
• Hierarchy of formal grammar based on level of complexity
• Grammars use phrase structure rules (Rewrite rules)
• General framework - Generative grammar
• Any grammar that uses a set of rules to specify or generate all
and only grammatical (well-formed) sentences in a language.
• Each sentence in a language has two levels of representations:
• Deep structure
• Surface structure
• Mapping from deep to surface structure is carried out by
transformations.
• Chomsky argued that an utterance is the surface representation of a
'deeper structure' representing its meaning.
• The deep structure can be transformed in a number of ways to yield
many different surface-level representations.
• Sentences with different surface-level representations having the same
meaning, share a common deep-level representation.
Pooja plays veena.
Veena is played by Pooja.
• Both the sentences are being generated from the same 'deep structure'
in which the deep subject is Pooja and the deep object is the veena.
• Transformational grammar has three components:
1. Phrase structure grammar
2. Transformational rules
3. Morphophonemic rules-These rules match each sentence representation to a
string of phonemes.
• Phrase structure grammar consists of rules that generate natural
language sentences and assign a structural description to them.
• The set of rules:
S ---> NP + VP
VP ---> V+ NP
NP ---> Det + Noun
V ---> Aux + Verb
Det ---> the, a, an, ...
Verb ---> catch, write, eat…
Noun ---> police, snatcher….
Aux ---> will, is, can…
• Sentences that can be generated using these rules are termed
grammatical. The structure assigned by the grammar is a constituent
structure analysis of the sentence.
• The second component is a set of transformation rules, which
transform one phrase-maker into another phrase-marker.
These rules are applied on the terminal string generated by
phrase structure rules.
• e.g., an active sentence into passive one.
NP₁ - Aux - V - NP₂ → NP2 - Aux + be + en - V - by + NP1
• This transformation involves addition of strings 'be' and 'en'
and certain re-arrangements of the constituents of a sentence.
• Transformational rules can be obligatory or optional.
• Obligatory transformation - that ensures agreement in number
of subject and verb, etc.
• Optional transformation - modifies the structure of a sentence
while preserving its meaning.
• Morphophonemic rules match each sentence representation to
a string of phonemes.
 The police will catch the snatcher.
• The passive transformation rules will convert the sentence into:
• The + snatcher + will + be + en + catch + by + police; as per [NP₁
- Aux - V - NP₂ → NP2 - Aux + be + en - V - by + NP1].
• Another transformational rule will then reorder 'en + catch' to
'catch + en' and one of the morphophonemic rules will convert
'catch + en' to 'caught’.
• The noun phrase is not always as simple as in sentence. It may
contain other embedded structures, such as adjectives,
modifiers, relative clause, etc.
• The problem in the specification of appropriate phrase
structure rules occurs because these phenomena cannot be
localized at the surface structure level.
• “Vicki laughed.” “Vicki” is a NP. “laughed” is a VP.
• “The woman laughed.” “The woman” is the NP ; “laughed” is
the VP.
• “Vicki who lives near me laughed.” “Vicki who lives near me” is
the NP; “laughed” is the VP.
• “Vicki who lives near me laughed loudly.” “Vicki who lives near
me” is the NP; “laughed loudly” is the VP.
 PROCESSING INDIAN LANGUAGES
There are some differences between Indian languages and English. So, there is a
differences in their processing.
• Indic scripts have a non-linear structure.

• Indian languages have SOV (Subject-Object-Verb) as the default sentence structure.

• Indian languages have a free word order, ie., words can be moved freely within a
sentence without changing the meaning of the sentence.

• Spelling standardization is more subtle in Hindi than in English.

• Indian languages have a relatively rich set of morphological variants.

• Indian languages make extensive and productive use of complex predicates (CPs).

• Indian languages use post-position (Karakas) case markers instead of prepositions.

• Indian languages use verb complexes consisting of sequences of verbs, e.g., गा रहा
है (ga raha hai - singing) and खेल रही है (khel rahi hai - playing). The auxiliary
verbs in this sequence provide information about tense, aspect, modality, etc.
• Except for the direction in which its
script is written, Urdu is closely
related to Hindi. Both share similar
phonology, morphology, and syntax.
Both are free-word-order languages
and use post-positions.
• They also share a large amount of
their vocabulary. Urdu vocabulary
comes from Persian and Arabic, while
Hindi borrows much of its vocabulary
from Sanskrit.
• Paninian grammar provides a
framework for Indian language
models. These can be used for
computation of Indian languages. The
grammar focuses on extraction of
Karaka relations from a sentence.
 NLP APPLICATIONS
• Machine translation - It involves the complete linguistic analysis
of a natural language sentence, and linguistic generation of an
output sentence.
• Speech Recognition
• Speech Synthesis - utterance of natural language sentences
• Natural Language Interfaces to Databases - querying a
structured database using natural language sentences.
• Information Retrieval – use of indexing (stop word elimination,
stemming, phrase extraction, etc.), word sense disambiguation,
query modification, and knowledge bases. Lexical resources are
WordNet, LDOCE (Longman Dictionary of Contemporary
English) and Roget's Thesaurus.
• Information Extraction
• Question Answering
• Text Summarization
 LANGUAGE MODELLING
• The domain of language is quite vast.
• To handle such a large number of sentences, we have to create a
model of the domain, which can subsequently be simplified and
handled computationally.
• To create a general model of any language is a difficult task.
• Natural language is a complex entity and in order to process it
through a computer-based program, need to build a model of it.
This is known as language modelling.
• There are 2 approaches for language modelling.
• One is to define a grammar that can handle the language. The
other is to capture the patterns in a grammar language
statistically. [Grammar-based model and Statistical language
model]
• These include lexical functional grammar, government and
binding, Paninian grammar, and gram-based model.
• Study the grammar of various languages, compare them, and if
possible, arrive at reasonable models that facilitate our
understanding of the problem and designing of natural-
language tools.
• A grammar-based language model attempts to distinguish a
grammatical sentence from a non-grammatical one, whereas a
Probabilistic language model attempts to identify a sentence
based on a probability measure, usually a maximum likelihood
estimate.
• The Grammar-based language model, attempts to represent the
syntactic structure of language. Grammar consists of hand-
coded rules defining the structure and ordering of various
constituents (phrase, sentence, etc.).
• The statistical approach creates a language model by training it
from a corpus.
• SLM is one of the fundamental tasks, in many applications like
speech recognition, spelling correction, handwriting recognition,
machine translation, also for in information retrieval, text
summarization, and question answering.
 STATISTICAL LANGUAGE MODEL
• A statistical language model is a probability distribution P(s) over all possible word
sequences (or any other linguistic unit like words, sentences, paragraphs,
documents, or spoken utterances).

• The goal of a statistical language model is to estimate the probability (likelihood)

of a sentence.

• N-gram Model

• Decomposing sentence probability into a product of conditional probabilities using

the chain rule.

P(s) = P(w1, w2, w3,...,wn)

P(s) = P(w1) P(w2/ w1) P(w3 / w1 w2) P(w4 / w1 w2 w3)......... P(wn / w1 w2 ……wn-1 )

P(s) = ς𝑛𝑖=1 𝑃(wi / hi)

where hi, is history of word wi, defined as [w1 w2 ……wi-1]

• Here we calculate the probability of a word, given the
sequence of words preceding it.
• This model simplifies the task by approximating the
probability of a word given all the previous words by the
conditional probability given previous (n-1) words only.
P(wi / hi) ≈ P(wi / wi-n+1 .wi-1 )
• Thus, an n-gram model calculates P(wi / hi) by modelling
language as Markov model of order n-1, i.e., by looking at
previous (n-1) words only.
• A model that limits the history to the previous one word
only, is termed a bi-gram (n=1) model.
• A model that conditions the probability of a word to the
previous two words, is called a tri-gram (n=2) model.
• Using bi-gram and tri-gram estimate, the probability of a sentence can be
calculated as:

P(s) ≈ ς𝑛𝑖=1 P(wi / wi−1) and P(s) ≈ ς𝑛𝑖=1 P(wi / wi−2 .wi−1)

• For example, the bi-gram approximation of P(east / The Arabian knights are
fairy tales of the) is P(east/the), whereas a tri-gram approximation is
P(east/of the).

• A special word (pseudo word) <s> is introduced to mark the beginning of the
sentence in bi-gram estimation. The probability of the first word in a
sentence is conditioned on <s>.

• In tri-gram estimation, we introduce two pseudo-words <s1> and <s2>.

• How to estimate the probabilities. This is done by training the n-gram model
on the training corpus.

• We estimate n-gram parameters using the maximum likelihood estimation

(MLE) technique, i.e., using relative frequencies.
• We count a particular n-gram in the training corpus and divide it
by the sum of all n-grams that share the same prefix.

C(wi−n+1,……, wi−1,wi)
P(wi / wi-n+1,……, wi-1) = σ
𝑤 C(wi−n+1,……, wi−1,w)

• The sum of all n-grams that share first (n-1) words is equal to the
count of the common prefix [wi-n+1,……, wi-1].

C(wi−n+1,……, wi−1,wi)
P(wi / wi-n+1,……, wi-1) =
C(wi−n+1,……, wi−1)
• The model parameter we get using these estimates, maximizes
the probability of the training set T given the model M, ie., P(T/M).
• The frequency with which a word occurs in a text may not be the
same as in the training set; this model only provides the most
likely solution.
 world
I am the king of the ……….
• 1-gram(Unigram)-----(i)(am)(the)(king) …..
• 2-gram(bigram)-----(I am)(am the)(the king)….
• 3-gram(trigram)------(I am the)(am the king)….
 Corpus
The girl bought a chocolate
The boy ate the chocolate
Vocabulary
The girl bought a toy
{the, girl, bought, a,
chocolate, boy, ate, The girl played with the toy
toy, played, with}
Input: The girl …?….
 Training set:
The Arabian Knights
These are the fairy tales of the east
The stories of the Arabian knights are translated in many languages
• Bi-gram model:
• P(the/<s>) = 0.67 P(Arabian/the)=0.4 P(knights/Arabian) = 1.0
P(the/are) = 0.5 P(fairy/the) = 0.2 P(tales/fairy) =1.0
P(of/tales) = 1.0 P(the/of) = 1.0 P(east/the) = 0.2
P(stories/the) =0.2 P(of/stories) =1.0 P(are/knights) =0.5
P(translated/are) = 0.5 P(in/translated) = 1.0 P(many/in) = 1.0
P(languages/many) = 1.0 P(are/these) = 1.0
• Test sentence(s): The Arabian knights are the fairy tales of the east.
• P(The/<s>) x P(Arabian/the) x P(Knights/Arabian) x P(are/knights) x P(the/are)
x P(fairy/the) x P(tales/fairy) x P(of/tales) x P(the/of) x P(east/the)
= 0.67 x 0.4 x 1.0 x 0.5 x 0.5 x 0.2 x 1.0 x 1.0 x 1.0 x 0.2 = 0.00267
• As each probability is less than 1, multiplying the probabilities
might cause a numerical underflow, particularly in long
sentences.

• To avoid this, calculations are made in log space, where a

calculation corresponds to adding log of individual probabilities
and taking antilog of the sum.

• The n-gram model suffers from data sparseness problem.

• An n-gram that does not occur in the training data is assigned

zero probability, so that even a large corpus has several zero
entries in its bi-gram matrix.

• There are several long-distance dependencies in natural

language sentences, which this model fails to capture.
• A number of smoothing techniques have been developed to
handle the data sparseness problem, the simplest of these
being add-one smoothing.
• Smoothing is to re-evaluating zero-probability or low-
probability n-grams and assigning them non-zero values.
• This is to make distributions more uniform by moving the
extreme probabilities towards the average.
• Add-one probabilities
C(wi−n+1,……, wi−1,wi)+1
P(wi / wi-n+1,……, wi-1) =
C(wi−n+1,……, wi−1)+𝑉
where V is the vocabulary size, i.e., size of the set of all the words
being considered.
 Paninian Framework
• The Paninian Grammar-based model (Kiparsky 1982, Bharti et
al. 1995).
• Paninian grammar (PG) was written by Panini in 500 BC in
Sanskrit (the original text being titled Asthadhyayi), the
framework can be used for other Indian languages and possibly
some Asian languages as well.
• Unlike English, Asian languages are SOV (Subject-Object-Verb)
ordered and inflectionally rich.
• The inflections provide important syntactic and semantic cues
for language analysis and understanding.
• There are many complexities of Indian languages which are yet
to be explained through this or other models.
• Indian languages have traditionally used oral communication for
knowledge propagation.
• Oral traditions have given rise to a morphologically rich language.
• In Sanskrit, have the flexibility to allow word groups representing
subject, object, and verb to occur in any order. In Hindi, we can
change the position of subject and object.
(a) मााँ बच्चे को खाना वेती है । [Maan Bachche ke khanaa detii hai]

Mother child to food give-(s)

Mother gives food to the child.

(b) बच्चे को मााँ खाना दे ती है । [Bachche ko Maan khanaa detii hai]

Child to mother food give-(s)

Mother gives food to the child.

• The auxilary verbs follow the main verb. In Hindi, they remain
as separate words, whereas in south Indian (Dravidian)
languages, they combine with the main verb. For example:
खा रहा है करता रहा है वह चला
khaa raha hai kartaa rahaa hai He moved
eat-ing doing been has वह चल ददया
eating has been doing He move given
He moved (started the action)

उसने खाना खाया। उसने खाना खा ललया।

Usne khanaa khaayaa Usne khaanaa kha liyaa
He (Subj) food ate He (Subj) food eat taken
He ate food He ate food (completed the action)
• In Indian languages, the nouns are followed by post-positions instead
of prepositions. They generally remain as separate words in Hindi,
except in the case of pronouns, for example
रे खा के पिता उसके पिता
Rekha ke pita Uske pita
Rekha of father Her (His) father
Father of Rekha
• A new framework for handling Indian languages.
• Verb groups are formed differently in Indo Aryan and Dravidian
languages.
• Sanskrit is very different from the other Indian languages as it has
five tenses and three numbers, and only one time aspect in each
tense. Hence, the translation of 'He goes' and 'He is going’ is the
same in Sanskrit.
• Hindi is unique in the sense that it has no neuter gender. All nouns
are categorized as feminine, and the verb form must have a gender
agreement with the subject.
 ताला खो गया चाभी खो गयी
Taalaa kho gayaa Chaabhii kho gayeee
Lock lose (past) key lose (past)
The lock was lost. The key was lost.

Layered Representation in PG
• The Government and Binding (GB) theory
represents 3 syntactic levels: deep
structure, surface structure, and logical
form (LF), where the LF is nearer to
semantics.
• This theory tries to resolve all language
issues at syntactic levels only.
• Paninian grammar framework is said to be
syntactico-semantic, that is, one can go
from surface layer to deep semantics by
passing through intermediate layers.
• Vibhakti refers to word (noun, verb, or other) groups
based either on case endings, or post-positions, or
compound verbs, or main and auxiliary verbs, etc.
• Word groups are formed based on various kinds of
markers. These markers are language-specific, but all
Indian languages can be represented at the Vibhakti
level.
• Karaka (pronounced Kaaraka) literally means Case.
Paninian Grammar has its own way of defining Karaka
relations.
• These relations are based on the way the word groups
participate in the activity denoted by the verb group.
• Complexities arise because of the absence of
inflections, multiple categories and meanings of the
words, and the presence of a large number of
exceptions.
• These exceptions are not only applicable on stated
rules but also on future rules. Such forward and
backward chaining makes actual implementation
difficult.
• The resolution of ambiguities is a contentious issue,
often left to the listener.
• Multiple-meaning texts are abundant in Indian
literature as seen in the hundreds of interpretations
of the epics.
 Karaka Theory
• Karaka theory is the central theme of PG framework.
• Karaka relations are assigned based on the roles, which
are reflected in the case markers and post-position
markers (parsargs).
• These relations are similar to case relations in English,
but the types of relations are defined in a different
manner and the richness of the case endings found in
Indian languages has been used to its advantage.
• The various Karakas, such as Karta (subject), Karma
(object), Karana (instrument), Sampradana
(beneficiary), Apadan (separation), and Adhikaran
(locus).
 मााँ बच्ची को आाँगन में हाथ से रोटी खखलाती है ।
Maan bachchi ko aangan mein haath se rotii khilaatii hei
Mother child-to courtyard-in hand-by bread feed (s).
The mother feeds bread to the child by hand in the courtyard.
• Karta is defined as the noun group which is most independent.
Karta has generally 'ne' or 'φ' case marker. It is an independent
entity in the activity denoted by the main verb.
• In above example, 'maan' (mother) is the Karta. The concept of
Karta is different from the 'agent' concept in the sense that Karta
can also take up the role of experience, e.g..
मझ
ु से रहा न गया। Mujhse rahaa na gayaa
Me hold not passive
I could not hold myself.
 मााँ बच्ची को आाँगन में हाथ से रोटी खखलाती है ।
Maan bachchi ko aangan mein haath se rotii khilaatii
hei
• maan (mother) is the Karta
• rotii (bread) is the Karma
• haath (hand) is the Karan
• bachchi (child) is the Sampradan
• aangan (courtyard) is the Adhikaran
• ‘Karma' is similar to object and is the locus of the
result of the activity.
• rotii (bread) is the Karma.
• when the Karta is the experiencer, it (she) is also
the locus of the result.
• Thus, the locus of the result is only when it is
different from Karta termed Karma.
• Karma generally has 'φ' or 'KO' case marker.
• 'Karan' (instrument), which is a noun group
through which the goal is achieved.
• haath (hand) is the Karan. It has the marker
dwara (by) or se.
• 'Sampradan' is the beneficiary of the activity, e.g., bachchi
(child). It takes the marker ko (to) or ke liye (for).
• 'Apaadaan' denotes separation and the marker is attached to
the part that serves as a reference point (being stationary),
मााँ ने थाली से खाना उठाकर बच्चे को ददया । Maan ne thaali se
khana uthakar bachche ko diyaa.
Mother plate from food taking up child-to gave.
The mother gave food to the child taking it up from the plate.
• Here thaali is the Apaadan.
• 'Adhikaran' is the locus (support in space or time) of Karta or
Karma. aangan (courtyard) is the Adhikaran.
• As these six relations are not sufficient to capture all possible
relations, various others such as 'Sambandh' (relation) and
"Tadarthya' (purpose) have also been tried.
 Issues in Paninian Grammar
The two problems challenging linguists are:
(i) Computational implementation of PG, and
(ii) Adaptation of PG to Indian, and other similar languages.
• In Bharati, et al. (1995) research work they spoke the PG is a multilayered
implementation. The approach is named 'Utsarga-Apvada' (default-
exception), where rules are arranged in multiple layers in such a way that
each layer consists of rules which are in exception to rules in the higher layer.
As we go down the layer, more particular information is derived. Rules may
be represented in the form of charts.
• Another difficulty arises when mapping between the Vibhakti and the
semantic relation is not one to one.
• Two different Vibhakti can represent the same relation, or the same Vibhakti
can represent different relations in different contexts. The strategy to
disambiguate the various senses of words, or word groupings, are still the
challenging issues.
• As the system of rules is different in different languages, the framework
requires adaptations to tackle various applications in various languages.
 Questions?
1. What is NLP? How it has been originated and what are the challenges of NLP?
2. What is NLP? List and explain different levels of processing involved in it.
3. Explain semantic and syntactic levels of text analysis.
4. Explain the challenges of NLP.
5. Explain in detail about Transformational grammar.
6. List and explain in detail applications of NLP.
7. Describe how to process Indian Languages.
8. List the key differences between Processing Indian Languages against English language.
9. Differentiate between grammar based model and statistical based model.
10. Explain n-gram modelling of natural languages. Find the probability of the test sentence
S4 in the following training set
S1: The Arabian knights
S2: These are the fairy tales of the east
S3: The stories of the Arabian Knights are translated in many languages
S4: The Arabian Knights are the fairy tales of the east
11. Determine the probability for training set corpus, predict the test case sentences
probability and choose the highest probability sentence among two by the use of
bigram model.
Training Set:
<s> I am Chintu
<s> I like College
<s> Do Chintu like College
<s> Chintu I am
<s> Do I like Chintu
<s> Do I like College
<s> I do like Chintu
Testing Set:
I like College
Do I like Chintu
12. Determine the probability of words and sentence using add one smoothing.
Training Set:
I love India
India is my country
I feel proud
Testing Set:
I love India and feel proud
my country
13. What are the different approaches for language modeling? Explain.
14. What is statistical language model and explain features of n-gram model?
15. What is data sparseness problem in n-gram model? Explain.
16. Briefly explain the problems associated with n-gram model and the techniques to
resolve problems.
17. Using Bi-gram model, find the probability of the sentence “They play in a garden” for
the following training set data;
There is a big garden
Children play in the garden
They play inside beautiful garden
18. Consider the Training set below:
The girl cat
The boy and the cat
The cat and the girl
The girl and the cat
Girl and the boy
i. Estimate the probability of the sentence “The boy and the girl cat” using a model that limits
the history to the previous one word only.
ii. Estimate the probability of the sentence “The boy and the girl cat” using a model that limits
the history to the previous two words.