NLP

Syllabus

What is Natural Language Processing?

Natural Language Processing is a part of artificial intelligence that aims to teach the
human language with all its complexities to computers. This is so that machines can
understand and interpret the human language to eventually understand human
communication in a better way. Natural Language Processing is a cross among many
different fields such as artificial intelligence, computational linguistics,
human-computer interaction, etc. There are many different methods in NLP to
understand human language which include statistical and machine learning methods.
And why is Natural Language Processing important, you wonder? Well, it allows
computers to understand human language and then analyze huge amounts of
language-based data in an unbiased way. This is the reason that Natural Language
Processing has many diverse applications these days in fields ranging from IT to
telecommunications to academics. So, let’s see these applications now.
Applications of Natural Language Processing

1. Chatbots

Chatbots are a form of artificial intelligence that are programmed to interact with
humans in such a way that they sound like humans themselves. Depending on the
complexity of the chatbots, they can either just respond to specific keywords or they
can even hold full conversations that make it tough to distinguish them from humans.
Chatbots are created using Natural Language Processing and Machine Learning,
which means that they understand the complexities of the English language and find
the actual meaning of the sentence and they also learn from their conversations with
humans and become better with time. Chatbots work in two simple steps. First, they
identify the meaning of the question asked and collect all the data from the user that
may be required to answer the question. Then they answer the question appropriately.

2. Autocomplete in Search Engines

Have you noticed that search engines tend to guess what you are typing and
automatically complete your sentences? For example, On typing “game” in Google,
you may get further suggestions for “game of thrones”, “game of life” or if you are
interested in maths then “game theory”. All these suggestions are provided using
autocomplete that uses Natural Language Processing to guess what you want to ask.
Search engines use their enormous data sets to analyze what their customers are
probably typing when they enter particular words and suggest the most common
possibilities. They use Natural Language Processing to make sense of these words and
how they are interconnected to form different sentences.

3. Voice Assistants

These days voice assistants are all the rage! Whether its Siri, Alexa, or Google
Assistant, almost everyone uses one of these to make calls, place reminders, schedule
meetings, set alarms, surf the internet, etc. These voice assistants have made life much
easier. But how do they work? They use a complex combination of speech
recognition, natural language understanding, and natural language processing to
understand what humans are saying and then act on it. The long term goal of voice
assistants is to become a bridge between humans and the internet and provide all
manner of services based on just voice interaction. However, they are still a little far
from that goal seeing as Siri still can’t understand what you are saying sometimes!

4. Language Translator

Want to translate a text from English to Hindi but don’t know Hindi? Well, Google
Translate is the tool for you! While it’s not exactly 100% accurate, it is still a great
tool to convert text from one language to another. Google Translate and other
translation tools as well as use Sequence to sequence modeling that is a technique in
Natural Language Processing. Earlier, language translators used Statistical machine
translation (SMT) which meant they analyzed millions of documents that were
already translated from one language to another (English to Hindi in this case) and
then looked for the common patterns and basic vocabulary of the language. However,
this method was not that accurate as compared to Sequence to sequence modeling.

5. Sentiment Analysis

Almost all the world is on social media these days! And companies can use sentiment
analysis to understand how a particular type of user feels about a particular topic,
product, etc. They can use natural language processing, computational linguistics, text
analysis, etc. to understand the general sentiment of the users for their products and
services and find out if the sentiment is good, bad, or neutral. Companies can use
sentiment analysis in a lot of ways such as to find out the emotions of their target
audience, to understand product reviews, to gauge their brand sentiment, etc.

6. Grammar Checkers

Grammar and spelling is a very important factor while writing professional reports for
your superiors even assignments for your lecturers. After all, having major errors may
get you fired or failed! That’s why grammar and spell checkers are a very important
tool for any professional writer. They can not only correct grammar and check
spellings but also suggest better synonyms and improve the overall readability of your
content. And guess what, they utilize natural language processing to provide the best
possible piece of writing! The NLP algorithm is trained on millions of sentences to
understand the correct format.

7. Email Classification and Filtering

Emails are still the most important method for professional communication. However,
all of us still get thousands of promotional Emails that we don’t want to read.
Thankfully, our emails are automatically divided into 3 sections namely, Primary,
Social, and Promotions which means we never have to open the Promotional section!
But how does this work? Email services use natural language processing to identify
the contents of each Email with text classification so that it can be put in the correct
section.

NLP Phases
Natural Language Processing (NLP) is a field within artificial intelligence that allows
computers to comprehend, analyze, and interact with human language effectively. The
process of NLP can be divided into five distinct phases: Lexical Analysis, Syntactic
Analysis, Semantic Analysis, Discourse Integration, and Pragmatic Analysis. Each
phase plays a crucial role in the overall understanding and processing of natural
language.
1. Lexical Analysis (Tokenization)

●​ Breaks text into words, tokens, or sentences​

●​ Removes punctuation and splits text​

●​ Example:​
“I love NLP!” → ["I", "love", "NLP"]

2. Syntactic Analysis (Parsing)

●​ Checks grammar and sentence structure​

●​ Builds a syntax tree​

●​ Example: Identifies nouns, verbs, etc., and how they relate

3. Semantic Analysis

●​ Extracts meaning of words and phrases​

●​ Detects word sense, synonyms, etc.​

●​ Example:​
“Bank” → (financial institution or riverbank?) based on context

4. Discourse Integration
 ●​ Considers meaning across multiple sentences​

●​ Understands pronouns, references, context​

●​ Example:​
"John went to the store. He bought milk." → “He” = John

5. Pragmatic Analysis

●​ Focuses on real-world meaning and intent​

●​ Considers speaker's intent, tone, emotion, etc.​

●​ Example:​
“Can you open the door?” → Not a question, it’s a request

NLP Approaches

1. Rule-Based Approach
●​ Uses manually written grammar rules and dictionaries​

●​ Based on linguistic knowledge​

●​ Good for: structured, predictable text​

●​ Example: If a sentence contains "is", "are", etc., mark it as a verb​

✅❌Pros: Easy to understand, interpretable​

Cons: Not scalable, fails with ambiguity or informal language

🧠 2. Statistical Approach
●​ Uses probability and machine learning​

●​ Trains on large datasets to make predictions​

 ●​ Example: Part-of-speech tagging using Hidden Markov Models (HMMs)​

✅❌Pros: Learns from data, better handling of variability​

Cons: Needs a lot of labeled data, less explainable

🤖 3. Machine Learning Approach

●​ Uses ML algorithms (e.g., Naive Bayes, SVM, Logistic Regression) for
NLP tasks like:​

○​ Spam detection​

○​ Sentiment analysis​

○​ Text classification​

✅❌Pros: General-purpose, customizable​

Cons: Still requires feature engineering

🧠💬 4. Deep Learning Approach (Neural Networks)

●​ Uses Neural Networks and models like:​

○​ RNNs, LSTMs (for sequences)​

○​ CNNs (for sentence-level tasks)​

○​ Transformers (like BERT, GPT)​

✅❌Pros: State-of-the-art results, handles context well​

Cons: Requires high computational power & lots of data

🧪 5. Hybrid Approach
●​ Combines rule-based + statistical or ML​

●​ Example: Use rules for cleaning, ML for classification​

✅❌Best of both worlds, more flexible​
Can be complex to implement

Machine Learning for NLP

Machine Learning helps computers learn patterns from text data and perform
various language tasks without being explicitly programmed.

🔑 Common NLP Tasks Using ML

🧩 Task 📌 Description
Text Classification Categorize text (e.g., spam detection,
sentiment analysis)

Named Entity Recognition Find names, places, dates in text

(NER)

Part-of-Speech Tagging Identify nouns, verbs, adjectives, etc.

Language Translation Translate between languages

Text Summarization Generate summaries of long texts

Speech Recognition Convert spoken language to text

Chatbots/Virtual Assistants Understand and respond to user input

⚙️ Machine Learning Algorithms Used in NLP

1. Naive Bayes

●​ Used for text classification (e.g., spam filtering)​

 ●​ Based on word probabilities​

2. Support Vector Machines (SVM)

●​ Classifies texts with hyperplanes; works well for high-dimensional data​

3. Logistic Regression

●​ Good for binary or multi-class classification​

4. Decision Trees / Random Forest

●​ Used for interpretable models​

5. k-Nearest Neighbors (k-NN)

●​ Based on similarity between documents​

📈 How Machine Learning Works in NLP

1.​ Collect Data – Emails, tweets, reviews, articles, etc.​

2.​ Preprocess Text – Tokenization, removing stopwords, stemming​

3.​ Convert Text to Numbers –​

○​ Bag of Words​

○​ TF-IDF​

○​ Word Embeddings (Word2Vec, GloVe)​

4.​ Train ML Model – Feed numerical data to a learning algorithm​

5.​ Predict & Evaluate – Test model on unseen text​

What is Deep Learning?
Deep learning is when a computer learns by itself using lots of examples.

💬 What is NLP?
NLP (Natural Language Processing) helps computers understand human language —
like English, Hindi, or any other.

🤖 What is Deep Learning for NLP?

It means using smart computer models to:

●​ Read human language​

●​ Understand what it means​

●​ Reply like a human​

📱 Simple Examples:
●​ Talking to Alexa or Siri​

●​ Using Google Translate​

●​ Getting smart replies in Gmail​

●​ Chatting with ChatGPT​

🛠️ How it Works:
1.​ You give a sentence: “I am happy”​

2.​ The computer turns it into numbers​

3.​ The deep learning model learns patterns from thousands of similar sentences​

4.​ It gives an answer — like detecting emotion or translating it​

🧠 Popular Models:
 ●​ BERT – understands what you mean​

●​ GPT – talks like a human​

✅ One Line Summary:

Deep learning for NLP means teaching computers to understand and talk like
humans using smart learning and lots of examples.

NLP Libraries
NLP (Natural Language Processing) libraries are sets of tools and
algorithms that simplify and automate the process of analyzing,
processing, and understanding human language. They provide
pre-built functions and models that can be used for various NLP
tasks, such as text analysis, sentiment analysis, machine translation,
and more.

Popular NLP Libraries:

​ NLTK (Natural Language Toolkit):​
A comprehensive Python library for a wide range of NLP tasks, including
tokenization, stemming, tagging, parsing, and semantic reasoning.
​ spaCy:​
A fast and efficient Python library for NLP tasks, particularly suited for
semantic analysis and large-scale text processing.
​ Hugging Face Transformers:​
A powerful library for working with pre-trained transformer models,
often used for tasks like text classification, question answering, and
translation.
​ TextBlob:​
A simplified interface for NLP tasks, built on top of NLTK, offering a
streamlined API for common NLP functions.
​ Gensim:​
Focuses on topic modeling and document similarity, allowing users to
extract insights from large text corpora.
​ Pattern:​
A comprehensive library with tools for sentiment analysis, SVM,
clustering, and more.
​ Deeplearning4j:​
A Java library specifically designed for deep learning in NLP, providing
tools for recurrent neural networks, convolutional neural networks, and
transformers.
​ FastText:​
A library developed by Facebook AI Research, known for its speed and
ability to handle text classification and word embeddings.
​ Franc:​
A [Link] library for detecting the language of a given text.
​ Indic NLP Library:​
A Python library specifically designed for processing text in Indian
languages, providing features like text normalization, script identification,
and translation.

Key Functions of NLP Libraries:

●​ Tokenization: Breaking down text into individual words or units.
●​ Stemming and Lemmatization: Reducing words to their root form.
●​ Part-of-Speech Tagging: Identifying the grammatical role of each word
in a sentence.
●​ Named Entity Recognition (NER): Identifying and classifying entities
like names, dates, and locations in text.
●​ Sentiment Analysis: Determining the emotional tone of a piece of text.
●​ Text Summarization: Extracting the key information from a longer
piece of text.
●​ Machine Translation: Translating text between languages.

NLP libraries are essential tools for developers and researchers working
with natural language, enabling them to build applications that can
understand and interact with human language. They offer a wide range of
features and pre-trained models, simplifying the process of implementing
various NLP tasks
POS(Parts-Of-Speech) Tagging in NLP
One of the core tasks in Natural Language Processing (NLP) is Parts of Speech (PoS)
tagging, which is giving each word in a text a grammatical category, such as nouns,
verbs, adjectives, and adverbs. Through improved comprehension of phrase structure
and semantics, this technique makes it possible for machines to study and comprehend
human language more accurately.
In many NLP applications, including machine translation, sentiment analysis, and
information retrieval, PoS tagging is essential. PoS tagging serves as a link between
language and machine understanding, enabling the creation of complex language
processing systems and serving as the foundation for advanced linguistic analysis.

What is POS(Parts-Of-Speech) Tagging?

Parts of Speech tagging is a linguistic activity in Natural Language Processing (NLP)
wherein each word in a document is given a particular part of speech (adverb,
adjective, verb, etc.) or grammatical category. Through the addition of a layer of
syntactic and semantic information to the words, this procedure makes it easier to
comprehend the sentence’s structure and meaning.
In NLP applications, POS tagging is useful for machine translation, named entity
recognition, and information extraction, among other things. It also works well for
clearing out ambiguity in terms with numerous meanings and revealing a sentence’s
grammatical structure.

Default tagging is a basic step for the part-of-speech tagging. It is performed using the
DefaultTagger class. The DefaultTagger class takes ‘tag’ as a single argument. NN is
the tag for a singular noun. DefaultTagger is most useful when it gets to work with the
most common part-of-speech tag. that’s why a noun tag is
recommended .
Example of POS Tagging
Consider the sentence: “The quick brown fox jumps over the lazy dog.”
After performing POS Tagging:
●​ “The” is tagged as determiner (DT)
●​ “quick” is tagged as adjective (JJ)
●​ “brown” is tagged as adjective (JJ)
●​ “fox” is tagged as noun (NN)
●​ “jumps” is tagged as verb (VBZ)
●​ “over” is tagged as preposition (IN)
●​ “the” is tagged as determiner (DT)
●​ “lazy” is tagged as adjective (JJ)
●​ “dog” is tagged as noun (NN)

By offering insights into the grammatical structure, this tagging aids machines in
comprehending not just individual words but also the connections between them
inside a phrase. For many NLP applications, like text summarization, sentiment
analysis, and machine translation, this kind of data is essential.

Workflow of POS Tagging in NLP

The following are the processes in a typical natural language processing (NLP)
example of part-of-speech (POS) tagging:
 ●​ Tokenization: Divide the input text into discrete tokens, which are
usually units of words or subwords. The first stage in NLP tasks is
tokenization.
●​ Loading Language Models: To utilize a library such as NLTK or
SpaCy, be sure to load the relevant language model. These models offer
a foundation for comprehending a language’s grammatical structure
since they have been trained on a vast amount of linguistic data.
●​ Text Processing: If required, preprocess the text to handle special
characters, convert it to lowercase, or eliminate superfluous
information. Correct PoS labeling is aided by clear text.
●​ Linguistic Analysis: To determine the text’s grammatical structure, use
linguistic analysis. This entails understanding each word’s purpose
inside the sentence, including whether it is an adjective, verb, noun, or
other.
●​ Part-of-Speech Tagging: To determine the text’s grammatical structure,
use linguistic analysis. This entails understanding each word’s purpose
inside the sentence, including whether it is an adjective, verb, noun, or
other.
●​ Results Analysis: Verify the accuracy and consistency of the PoS
tagging findings with the source text. Determine and correct any
possible problems or mistagging.

Types of POS Tagging in NLP

Assigning grammatical categories to words in a text is known as Part-of-Speech (PoS)

tagging, and it is an essential aspect of Natural Language Processing (NLP). Different
PoS tagging approaches exist, each with a unique methodology. Here are a few typical
kinds:
1. Rule-Based Tagging
Rule-based part-of-speech (POS) tagging involves assigning words their respective
parts of speech using predetermined rules, contrasting with machine learning-based
POS tagging that requires training on annotated text corpora. In a rule-based system,
POS tags are assigned based on specific word characteristics and contextual cues.
Rule: Assign the POS tag “noun” to words ending in “-tion” or “-ment.”
Text: “The presentation highlighted the key achievements of the project’s
development.”
Rule based Tags:
●​ “The” – Determiner (DET)
 ●​ “presentation” – Noun (N)
●​ “highlighted” – Verb (V)
●​ “the” – Determiner (DET)
●​ “key” – Adjective (ADJ)
●​ “achievements” – Noun (N)
●​ “of” – Preposition (PREP)
●​ “the” – Determiner (DET)
●​ “project’s” – Noun (N)
●​ “development” – Noun (N)

2. Transformation Based tagging

Transformation-based tagging (TBT) is a part-of-speech (POS) tagging method that
uses a set of rules to change the tags that are applied to words inside a text. In
contrast, statistical POS tagging uses trained algorithms to predict tags
probabilistically, while rule-based POS tagging assigns tags directly based on
predefined rules.
To change word tags in TBT, a set of rules is created depending on contextual
information. A rule could, for example, change a verb’s tag to a noun if it comes after
a determiner like “the.” The text is systematically subjected to these criteria, and after
each transformation, the tags are updated.
When compared to rule-based tagging, TBT can provide higher accuracy, especially
when dealing with complex grammatical structures. To attain ideal performance,
nevertheless, it might require a large rule set and additional computer power.
Consider the transformation rule: Change the tag of a verb to a noun if it follows a
determiner like “the.”
Text: “The cat chased the mouse”.
Initial Tags:
●​ “The” – Determiner (DET)
●​ “cat” – Noun (N)
●​ “chased” – Verb (V)
●​ “the” – Determiner (DET)
●​ “mouse” – Noun (N)

Transformation rule applied:

Change the tag of “chased” from Verb (V) to Noun (N) because it follows the
determiner “the.”
Updated tags:
 ●​ “The” – Determiner (DET)
●​ “cat” – Noun (N)
●​ “chased” – Noun (N)
●​ “the” – Determiner (DET)
●​ “mouse” – Noun (N)

3. Statistical POS Tagging

Utilizing probabilistic models, statistical part-of-speech (POS) tagging is a computer
linguistics technique that places grammatical categories on words inside a text. If
rule-based tagging uses massive annotated corpora to train its algorithms, statistical
tagging uses machine learning.
In order to capture the statistical linkages present in language, these algorithms learn
the probability distribution of word-tag sequences. CRFs (conditional random fields)
and Hidden Markov Models (HMMs) are popular models for statistical point-of-sale
classification. The algorithm estimates the chance of observing a specific tag given
the current word and its context by learning from labeled samples during training.
The most likely tags for text that hasn’t been seen are then predicted using the trained
model. Statistical POS tagging works especially well for languages with complicated
grammatical structures because it is exceptionally good at handling linguistic
ambiguity and catching subtle language trends.
●​ Hidden Markov Model POS tagging: Hidden Markov Models (HMMs)
serve as a statistical framework for part-of-speech (POS) tagging in
natural language processing (NLP). In HMM-based POS tagging, the
model undergoes training on a sizable annotated text corpus to discern
patterns in various parts of speech. Leveraging this training, the model
predicts the POS tag for a given word based on the probabilities
associated with different tags within its context.​
Comprising states for potential POS tags and transitions between them,
the HMM-based POS tagger learns transition probabilities and
word-emission probabilities during training. To tag new text, the model,
employing the Viterbi algorithm, calculates the most probable sequence
of POS tags based on the learned probabilities.​
Widely applied in NLP, HMMs excel at modeling intricate sequential
data, yet their performance may hinge on the quality and quantity of
annotated training data.

Advantages of POS Tagging

There are several advantages of Parts-Of-Speech (POS) Tagging including:

 ●​ Text Simplification: Breaking complex sentences down into their
constituent parts makes the material easier to understand and easier to
simplify.
●​ Information Retrieval: Information retrieval systems are enhanced by
point-of-sale (POS) tagging, which allows for more precise indexing and
search based on grammatical categories.
●​ Named Entity Recognition: POS tagging helps to identify entities such
as names, locations, and organizations inside text and is a precondition
for named entity identification.
●​ Syntactic Parsing: It facilitates syntactic parsing, which helps with
phrase structure analysis and word link identification.

Disadvantages of POS Tagging

Some common disadvantages in part-of-speech (POS) tagging include:

●​ Ambiguity: The inherent ambiguity of language makes POS tagging
difficult since words can signify different things depending on the
context, which can result in misunderstandings.
●​ Idiomatic Expressions: Slang, colloquialisms, and idiomatic phrases can
be problematic for POS tagging systems since they don’t always follow
formal grammar standards.
●​ Out-of-Vocabulary Words: Out-of-vocabulary words (words not
included in the training corpus) can be difficult to handle since the
model might have trouble assigning the correct POS tags.
●​ Domain Dependence: For best results, POS tagging models trained on a
single domain should have a lot of domain-specific training data
because they might not generalize well to other domains.

Syntactic analysis

Syntactic analysis in Natural Language Processing (NLP) is the process of

analyzing the grammatical structure of sentences to understand their meaning.
It involves identifying the relationships between words and phrases in a
sentence, essentially determining how the sentence is grammatically
constructed. This analysis helps NLP systems accurately interpret the meaning
of text and generate more meaningful outputs.
 Key Aspects of Syntactic Analysis:
​ Parsing:​
This is a core component of syntactic analysis. It involves breaking down a sentence
into its constituent parts and representing them in a structured form, such as a parse
tree.
​ Grammatical Rules:​
Syntactic analysis relies on the rules of grammar to determine the structure of a
sentence.
​ Parts of Speech (POS) Tagging:​
Identifying the grammatical category of each word (e.g., noun, verb, adjective) is a
crucial step in syntactic analysis.
​ Parse Trees:​
These tree-like structures visually represent the grammatical relationships between
words and phrases in a sentence.

Benefits of Syntactic Analysis:

​ Improved NLP Accuracy:​
By understanding the grammatical structure of sentences, NLP systems can make more
accurate interpretations of text.
​ Enhanced Machine Translation:​
Syntactic analysis helps in translating between languages by understanding the
grammatical relationships in the source language.
​ Better Text Generation:​
Syntactic analysis enables NLP systems to generate more coherent and grammatically
correct text.
​ Information Extraction:​
Syntactic analysis can help identify key information within a sentence, such as the
subject, verb, and object.
​ Machine Learning in NLP:​
Syntactic analysis provides valuable data and features for training machine learning
models in NLP.

Types of Parsing:
​ Top-down parsing:​
Starts from the top (root node) of the parse tree and works down, attempting to
match the input sentence against the grammar rules.
​ Bottom-up parsing:​
Starts from the bottom (leaves) of the parse tree and works up, building the tree
structure by combining words and phrases.

In essence, syntactic analysis is a crucial step in NLP that enables machines to

understand and interpret human language in a more sophisticated way,
ultimately leading to more accurate and powerful NLP applications.

Context-Free Grammars (CFGs) Explained Simply

Context-Free Grammar (CFG) is a formal way of describing the structure of
languages. It’s commonly used in computer science and linguistics to define how
sentences or strings in a language can be formed.

1. What is Grammar?
A grammar consists of a set of rules (or productions) that explain how to construct
sentences in a language. It defines how words can be combined to form valid
phrases or sentences.

For example, a simple grammar for a sentence might say:

●​ A sentence consists of a noun phrase and a verb phrase.

●​ A noun phrase consists of a noun.
●​ A verb phrase consists of a verb and optionally a noun phrase.​

2. What is Context-Free?
The term "context-free" means that the rules are applied independently of the
context. In other words, the way a symbol can be replaced is not dependent on the
surrounding symbols.

For example:

●​ In the rule S → NP VP, you can replace S with NP VP no matter where it

appears.​

3. Key Components of CFG

1.​ Non-terminal symbols (e.g., S, NP, VP):

○​ These are placeholders for groups of words or phrases.
○​ They are defined by the production rules.
2.​ Terminal symbols (e.g., dog, runs):
○​ These are the actual words (or tokens) in the language.
○​ They appear in the final output.
3.​ Production rules:
○​ They define how non-terminals can be expanded into other
non-terminals or terminals.
○​ Example: S → NP VP means S can be replaced with a noun phrase
(NP) and a verb phrase (VP).
4.​ Start symbol:
○​ A special non-terminal symbol (often S) from which the grammar
starts generating sentences.

4. Example of a Simple Context-Free Grammar

Let’s look at a very simple CFG to describe a sentence.

Production Rules:

1.​ S → NP VP
2.​ NP → Det N
3.​ VP → V NP
4.​ Det → the | a
5.​ N → dog | cat
6.​ V → chases | eats

Start symbol: S

Explanation:

●​ S (Sentence) can be broken into a noun phrase (NP) and a verb phrase (VP).
 ●​ NP (Noun Phrase) consists of a determiner (Det) and a noun (N).
●​ VP (Verb Phrase) consists of a verb (V) and another NP.​

5. Generating Sentences Using the Grammar

1.​ Start with S.
2.​ Apply S → NP VP → Now we have NP VP.
3.​ Replace NP → Det N → Now we have Det N VP.
4.​ Replace Det → the and N → dog → Now we have the dog VP.
5.​ Replace VP → V NP → Now we have the dog V NP.
6.​ Replace V → chases and NP → Det N → Now we have the dog chases the cat.

Generated sentence: "The dog chases the cat."

6. Why is it Called Context-Free?

In a context-free grammar, the rules can be applied without any context. That
means you can replace a non-terminal (like S) with its production (like NP VP)
anywhere in a sentence, no matter what words are around it.

For example:

●​ In the rule S → NP VP, it doesn’t matter if S is at the start, middle, or end of

a sentence — the same rule applies everywhere.​

7. Why Use Context-Free Grammars?

●​ Parsing: CFGs are useful in building parsers that help computers understand
and process natural languages or programming languages.
●​ Language Design: CFGs help design the structure of programming languages
(like Java or Python) or natural languages.
●​ Efficiency: They allow for efficient parsing algorithms, like Earley’s
algorithm or CYK algorithm.​

Key Takeaways
●​ Context-Free Grammar (CFG) defines how to form sentences in a language
using production rules.
●​ Context-free means that the rules can be applied independently of the
surrounding context.
 ●​ Non-terminal symbols can be replaced by other symbols (either terminal or
non-terminal) based on the production rules.
●​ CFGs are widely used in compilers, parsing natural languages, and building
structured data.

Dependency Parsing
Dependency parsing is a natural language processing technique that is used to
analyze the grammatical structure of sentences. It is a type of syntactic parsing
that aims to identify the relationships, or dependencies, between words in a
sentence. The output of a dependency parser is typically a dependency tree or a
graph, which represents the relationships between the words in the sentence.
The process of dependency parsing involves identifying the syntactic
relationships between words in a sentence. This typically involves identifying the
subject, object, and other grammatical elements, and then determining the
relationships between them. The parser uses a set of grammatical rules and a
grammar model to analyze the sentence and construct a dependency tree or
graph.
Dependency parsing is an important step in natural language processing and is
used in a wide range of applications, such as natural language understanding,
machine translation, and text summarization.
Dependency parsing is different from constituency parsing, which aims to
identify the hierarchical structure of a sentence. Dependency parsing focuses on
the linear structure of the sentence and the relationships between words, while
constituency parsing focuses on the hierarchical structure of the sentence. Both
techniques have their own advantages and can be used together to better
understand a sentence.
Some challenges in Dependency Parsing are the handling of long-distance
dependencies, syntactic ambiguity, and the handling of idiomatic expressions,
which makes the parsing process more complex.
Applications of Dependency Parsing
Dependency parsing is a process of analyzing the grammatical structure of a
sentence by identifying the dependencies between the words in a sentence and
representing them as a directed graph.
The following are some of the applications of dependency parsing:
 1.​ Named Entity Recognition (NER) – It helps in identifying and
classifying named entities in a text such as people, places, and
organizations.
2.​ Part-of-Speech (POS) Tagging – It helps in identifying the parts of
speech of each word in a sentence and classifying them as nouns, verbs,
adjectives, etc.
3.​ Sentiment Analysis – It helps in determining the sentiment of a sentence
by analyzing the dependencies between the words and the sentiment
associated with each word.
4.​ Machine Translation – It helps in translating sentences from one
language to another by analyzing the dependencies between the words
and generating the corresponding dependencies in the target language.
5.​ Text Generation – It helps in generating text by analyzing the
dependencies between the words and generating new words that fit into
the existing structure.
6.​ Question Answering – It helps in answering questions by analyzing the
dependencies between the words in a question and finding relevant
information in a corpus.

Semantic Analysis – NLP

Semantic Analysis is a subfield of Natural Language Processing (NLP) that
attempts to understand the meaning of Natural Language. Understanding
Natural Language might seem a straightforward process to us as humans.
However, due to the vast complexity and subjectivity involved in human
language, interpreting it is quite a complicated task for machines. Semantic
Analysis of Natural Language captures the meaning of the given text while
taking into account context, logical structuring of sentences and grammar roles.

Parts of Semantic Analysis

Semantic Analysis of Natural Language can be classified into two broad parts:
1. Lexical Semantic Analysis: Lexical Semantic Analysis involves understanding
the meaning of each word of the text individually. It basically refers to fetching
the dictionary meaning that a word in the text is deputed to carry.
2. Compositional Semantics Analysis: Although knowing the meaning of each
word of the text is essential, it is not sufficient to completely understand the
meaning of the text.
For example, consider the following two sentences:
 ●​ Sentence 1: Students love GeeksforGeeks.
●​ Sentence 2: GeeksforGeeks loves Students.

Although both these sentences 1 and 2 use the same set of root words {student,
love, geeksforgeeks}, they convey entirely different meanings.
Hence, under Compositional Semantics Analysis, we try to understand how
combinations of individual words form the meaning of the text.

Tasks involved in Semantic Analysis

In order to understand the meaning of a sentence, the following are the major
processes involved in Semantic Analysis:
1.​ Word Sense Disambiguation
2.​ Relationship Extraction

Word Sense Disambiguation:

In Natural Language, the meaning of a word may vary as per its usage in
sentences and the context of the text. Word Sense Disambiguation involves
interpreting the meaning of a word based upon the context of its occurrence in a
text.
For example, the word ‘Bark’ may mean ‘the sound made by a dog’ or ‘the
outermost layer of a tree.’
Likewise, the word ‘rock’ may mean ‘a stone‘ or ‘a genre of music‘ – hence, the
accurate meaning of the word is highly dependent upon its context and usage in
the text.
Thus, the ability of a machine to overcome the ambiguity involved in identifying
the meaning of a word based on its usage and context is called Word Sense
Disambiguation.

Relationship Extraction:

Another important task involved in Semantic Analysis is Relationship

Extracting. It involves firstly identifying various entities present in the sentence
and then extracting the relationships between those entities.
For example, consider the following sentence:
Semantic Analysis is a topic of NLP which is explained on the GeeksforGeeks
blog. The entities involved in this text, along with their relationships, are shown
below.

Entities

Relationships

Elements of Semantic Analysis

Some of the critical elements of Semantic Analysis that must be scrutinized and
taken into account while processing Natural Language are:
●​ Hyponymy: Hyponymys refers to a term that is an instance of a generic
term. They can be understood by taking class-object as an analogy. For
example: ‘Color‘ is a hypernymy while ‘grey‘, ‘blue‘, ‘red‘, etc, are its
hyponyms.
●​ Homonymy: Homonymy refers to two or more lexical terms with the
same spellings but completely distinct in meaning. For example: ‘Rose‘
might mean ‘the past form of rise‘ or ‘a flower‘, – same spelling but
different meanings; hence, ‘rose‘ is a homonymy.
●​ Synonymy: When two or more lexical terms that might be spelt
distinctly have the same or similar meaning, they are called Synonymy.
For example: (Job, Occupation), (Large, Big), (Stop, Halt).
●​ Antonymy: Antonymy refers to a pair of lexical terms that have
contrasting meanings – they are symmetric to a semantic axis. For
example: (Day, Night), (Hot, Cold), (Large, Small).
●​ Polysemy: Polysemy refers to lexical terms that have the same spelling
but multiple closely related meanings. It differs from homonymy
because the meanings of the terms need not be closely related in the
case of homonymy. For example: ‘man‘ may mean ‘the human species‘
or ‘a male human‘ or ‘an adult male human‘ – since all these different
meanings bear a close association, the lexical term ‘man‘ is a polysemy.
 ●​ Meronomy: Meronomy refers to a relationship wherein one lexical term
is a constituent of some larger entity. For example: ‘Wheel‘ is a
meronym of ‘Automobile‘

Meaning Representation
While, as humans, it is pretty simple for us to understand the meaning of textual
information, it is not so in the case of machines. Thus, machines tend to represent
the text in specific formats in order to interpret its meaning. This formal
structure that is used to understand the meaning of a text is called meaning
representation.

Basic Units of Semantic System:

In order to accomplish Meaning Representation in Semantic Analysis, it is vital

to understand the building units of such representations. The basic units of
semantic systems are explained below:
1.​ Entity: An entity refers to a particular unit or individual in specific
such as a person or a location. For example GeeksforGeeks, Delhi, etc.
2.​ Concept: A Concept may be understood as a generalization of entities.
It refers to a broad class of individual units. For example Learning
Portals, City, Students.
3.​ Relations: Relations help establish relationships between various
entities and concepts. For example: ‘GeeksforGeeks is a Learning
Portal’, ‘Delhi is a City.’, etc.
4.​ Predicate: Predicates represent the verb structures of the sentences.

In Meaning Representation, we employ these basic units to represent textual

information.

Approaches to Meaning Representations:

Now that we are familiar with the basic understanding of Meaning

Representations, here are some of the most popular approaches to meaning
representation:
1.​ First-order predicate logic (FOPL)
2.​ Semantic Nets
3.​ Frames
 4.​ Conceptual dependency (CD)
5.​ Rule-based architecture
6.​ Case Grammar
7.​ Conceptual Graphs

Ambiguity in NLP and how to address them

Ambiguity in Natural Language Processing (NLP) happens because human
language can have multiple meanings. Computers sometimes confuse to
understand exactly what we mean unlike humans, who can use intuition and
background knowledge to infer meaning, computers rely on precise algorithms
and statistical patterns.
The sentence "The chicken is ready to eat" is ambiguous because it can be
interpreted in two different ways:
1.​ The chicken is cooked and ready to be eaten.
2.​ The chicken is hungry and ready to eat food.

This dual meaning arises from the structure of the sentence, which does not
clarify the subject's role (the eater or the one being eaten). Resolving such
ambiguities is essential for accurate NLP applications like chatbots, translation,
and sentiment analysis.
This article explores types of ambiguity in NLP and methods to address them
effectively.

Types of Ambiguity in NLP

The meaning of an ambiguous expression often depends on the situation, prior
knowledge, or surrounding words. For example: He is cool. This could mean he
is calm under pressure or he is fashionable depending on the context.

1. Lexical Ambiguity

Lexical ambiguity occurs when a single word has multiple meanings, making it
unclear which meaning is intended in a particular context. This is a common
challenge in language.
For example, the word "bat" can have two different meanings. It could refer to a
flying mammal, like the kind you might see at night. Alternatively, "bat" could also
refer to a piece of sports equipment used in games like baseball or cricket.

For computers, determining the correct meaning of such a word requires looking
at the surrounding context to decide which interpretation makes sense.

2. Syntactic Ambiguity

Syntactic ambiguity occurs when the structure or grammar of a sentence allows

for more than one interpretation. This happens because the sentence can be
understood in different ways depending on how it is put together.

For example, take the sentence, “The boy kicked the ball in his jeans.” This
sentence can be interpreted in two different ways: one possibility is that the boy was
wearing jeans and he kicked the ball while he was wearing them. Another
possibility is that the ball was inside the boy’s jeans, and he kicked the ball out of
his jeans.

A computer or NLP system must carefully analyze the structure to figure out
which interpretation is correct, based on the context.

3. Semantic Ambiguity

Semantic ambiguity occurs when a sentence has more than one possible meaning
because of how the words are combined. This type of ambiguity makes it unclear
what the sentence is truly trying to say.

For example, take the sentence, “Visiting relatives can be annoying.” This sentence
could be understood in two different ways. One meaning could be that relatives who
are visiting you are annoying, implying that the relatives themselves cause
annoyance. Another meaning could be that the act of visiting relatives is what is
annoying, suggesting that the experience of going to see relatives is unpleasant.

The confusion comes from how the words "visiting relatives" can be interpreted:
is it about the relatives who are visiting, or is it about the action of visiting? In
cases like this, semantic ambiguity makes it hard to immediately understand the
exact meaning of the sentence, and the context is needed to clarify it.

4. Pragmatic Ambiguity

Pragmatic ambiguity occurs when the meaning of a sentence depends on the

speaker’s intent, tone, or the situation in which it is said. This type of ambiguity
is common in everyday conversations, and it can be tricky for computers to
understand because it often requires knowing the broader context.

For example, consider the sentence, “Can you open the window?” In one situation,
it could be understood as a literal question asking if the person is physically able to
open the window. However, in another context, it could be a polite request, where
the speaker is asking the listener to open the window, even though they’re not
directly giving an order.

The meaning changes based on the tone of voice or social context, which is
something that is difficult for NLP systems to capture without understanding the
surrounding situation

5. Referential Ambiguity

Referential ambiguity occurs when a pronoun (like "he," "she," "it," or "they")
or a phrase is unclear about what or who it is referring to. This type of ambiguity
happens when the sentence doesn’t provide enough information to determine
which person, object, or idea the pronoun is referring to.

For example, consider the sentence, “Alice told Jane that she would win the prize.”
In this case, it’s unclear whether the pronoun "she" refers to Alice or Jane. Both
could be possible interpretations, and without further context, we can’t be sure. If
the sentence was about a competition, "she" could be referring to Alice, meaning
Alice is telling Jane that she would win the prize. However, it could also mean that
Alice is telling Jane that Jane would win the prize.

6. Ellipsis Ambiguity

Ellipsis ambiguity happens when part of a sentence is left out, making it unclear
what the missing information is. This often occurs in everyday conversation or
writing when people try to be brief and omit words that are understood from the
context.

For example, consider the sentence, "John likes apples, and Mary does too." The
word "does" is a shortened form of "likes apples," but it’s not explicitly stated. This
creates two possible interpretations:

1.​ Mary likes apples just like John, meaning both John and Mary enjoy
apples.
2.​ Mary likes something else (not apples), and the sentence is leaving out the
specific thing she likes.

Word Sense Disambiguation in NLP

Word sense disambiguation (WSD) in Natural Language Processing (NLP) is the
problem of identifying which “sense” (meaning) of a word is activated by the use
of the word in a particular context or scenario. In people, this appears to be a
largely unconscious process. The challenge of correctly identifying words in NLP
systems is common, and determining the specific usage of a word in a sentence
has many applications. The application of Word Sense Disambiguation involves
the area of Information Retrieval, Question Answering systems, Chat-bots, etc.
Word Sense Disambiguation (WSD) is a subtask of Natural Language Processing
that deals with the problem of identifying the correct sense of a word in context.
Many words in natural language have multiple meanings, and WSD aims to
disambiguate the correct sense of a word in a particular context. For example,
the word “bank” can have different meanings in the sentences “I deposited
money in the bank” and “The boat went down the river bank”.
WSD is a challenging task because it requires understanding the context in which
the word is used and the different senses in which the word can be used. Some
common approaches to WSD include:
1.​ Supervised learning: This involves training a machine learning model
on a dataset of annotated examples, where each example contains a
target word and its sense in a particular context. The model then learns
to predict the correct sense of the target word in new contexts.
2.​ Unsupervised learning: This involves clustering words that appear in
similar contexts together, and then assigning senses to the resulting
 clusters. This approach does not require annotated data, but it is less
accurate than supervised learning.
3.​ Knowledge-based: This involves using a knowledge base, such as a
dictionary or ontology, to map words to their different senses. This
approach relies on the availability and accuracy of the knowledge base.
4.​ Hybrid: This involves combining multiple approaches, such as
supervised and knowledge-based methods, to improve accuracy.

WSD has many practical applications, including machine translation,

information retrieval, and text-to-speech systems. Improvements in WSD can
lead to more accurate and efficient natural language processing systems.
The noun ‘star’ has eight different meanings or senses. An idea can be mapped to
each sense of the word. For example,

●​ “He always wanted to be a Bollywood star.” The word ‘star’ can be

described as “A famous and good singer, performer, sports player, actor,
personality, etc.”
●​ “The Milky Way galaxy contains between 200 and 400 billion stars”. In
this, the word star means “a big ball of burning gas in space that we view
as a point of light in the night sky.”

Difficulties in Word Sense Disambiguation

There are some difficulties faced by Word Sense Disambiguation (WSD).
●​ Different Text-Corpus or Dictionary: One issue with word sense
disambiguation is determining what the senses are because different
dictionaries and thesauruses divide words into distinct senses. Some
academics have proposed employing a specific lexicon and its set of
senses to address this problem. In general, however, research findings
based on broad sense distinctions have outperformed those based on
limited ones. The majority of researchers are still working on
fine-grained WSD.
●​ PoS Tagging: Part-of-speech tagging and sense tagging have been
shown to be very tightly coupled in any real test, with each potentially
constraining the other. Both disambiguating and tagging with words
are involved in WSM part-of-speech tagging. However, algorithms
designed for one do not always work well for the other, owing to the fact
that a word’s part of speech is mostly decided by the one to three words
immediately adjacent to it, whereas a word’s sense can be determined
by words further away.
Approaches for Word Sense Disambiguation
There are many approaches to Word Sense Disambiguation. The three main
approaches are given below:
1. Supervised: The assumption behind supervised approaches is that the context
can supply enough evidence to disambiguate words on its own (hence, world
knowledge and reasoning are deemed unnecessary).
Supervised methods for Word Sense Disambiguation (WSD) involve training a
model using a labeled dataset of word senses. The model is then used to
disambiguate the sense of a target word in new text. Some common techniques
used in supervised WSD include:
1.​ Decision list: A decision list is a set of rules that are used to assign a
sense to a target word based on the context in which it appears.
2.​ Neural Network: Neural networks such as feedforward networks,
recurrent neural networks, and transformer networks are used to
model the context-sense relationship.
3.​ Support Vector Machines: SVM is a supervised machine learning
algorithm used for classification and regression analysis.
4.​ Naive Bayes: Naive Bayes is a probabilistic algorithm that uses Bayes’
theorem to classify text into predefined categories.
5.​ Decision Trees: Decision Trees are a flowchart-like structure in which
an internal node represents feature(or attribute), the branch represents
a decision rule, and each leaf node represents the outcome.

2. Unsupervised: The underlying assumption is that similar senses occur in

similar contexts, and thus senses can be induced from the text by clustering word
occurrences using some measure of similarity of context. Using fixed-size dense
vectors (word embeddings) to represent words in context has become one of the
most fundamental blocks in several NLP systems. Traditional word embedding
approaches can still be utilized to improve WSD, despite the fact that they
conflate words with many meanings into a single vector representation. Lexical
databases (e.g., WordNet, ConceptNet, BabelNet) can also help unsupervised
systems map words and their senses as dictionaries, in addition to word
embedding techniques.
3. Knowledge-Based: It is built on the idea that words used in a text are related
to one another, and that this relationship can be seen in the definitions of the
words and their meanings. The pair of dictionary senses having the highest word
overlap in their dictionary meanings are used to disambiguate two (or more)
words. Lesk Algorithm is the classical algorithm based on Knowledge-Based
WSD. Lesk algorithm assumes that words in a given “neighborhood” (a portion
of text) will have a similar theme. The dictionary definition of an uncertain word
is compared to the terms in its neighborhood in a simplified version of the Lesk
algorithm.

What is Machine Translation?

Machine translation is a sub-field of computational linguistics that focuses on
developing systems capable of automatically translating text or speech from one
language to another. In Natural Language Processing (NLP), the goal of machine
translation is to produce translations that are not only grammatically correct but
also convey the meaning of the original content accurately.

What are the key approaches in Machine Translation?

In machine translation, the original text is decoded and then encoded into the
target language through two step process that involves various approaches
employed by language translation technology to facilitate the translation
mechanism.

1. Rule-Based Machine Translation

Rule-based machine translation relies on these resources to ensure precise

translation of specific content. The process involves the software parsing input
text, generating a transitional representation, and then converting it into the
target language with reference to grammar rules and dictionaries.

2. Statistical Machine Translation

Rather than depending on linguistic rules, statistical machine translation utilizes

machine learning for text translation. Machine learning algorithms examine
extensive human translations, identifying statistical patterns. When tasked with
translating a new source text, the software intelligently guesses based on the
statistical likelihood of specific words or phrases being associated with others in
the target language.

3. Neural Machine Translation (NMT)

A neural network, inspired by the human brain, is a network of interconnected

nodes functioning as an information system. Input data passes through these
nodes to produce an output. Neural machine translation software utilizes neural
networks to process vast datasets, with each node contributing a specific change
from source text to target text until the final result is obtained at the output node.

4. Hybrid Machine Translation

Hybrid machine translation tools integrate multiple machine translation models

within a single software application, leveraging a combination of approaches to
enhance the overall effectiveness of a singular translation model. This process
typically involves the incorporation of rule-based and statistical machine
translation subsystems, with the ultimate translation output being a synthesis of
the results generated by each subsystem.

Why we need Machine Translation in NLP?

Machine translation in Natural Language Processing (NLP) has several benefits,
including:
1.​ Improved communication: Machine translation makes it easier for
people who speak different languages to communicate with each other,
breaking down language barriers and facilitating international
cooperation.
2.​ Cost savings: Machine translation is typically faster and less expensive
than human translation, making it a cost-effective solution for
businesses and organizations that need to translate large amounts of
text.
3.​ Increased accessibility: Machine translation can make digital content
more accessible to users who speak different languages, improving the
user experience and expanding the reach of digital products and
services.
4.​ Improved efficiency: Machine translation can streamline the
translation process, allowing businesses and organizations to quickly
translate large amounts of text and improving overall efficiency.
5.​ Language learning: Machine translation can be a valuable tool for
language learners, helping them to understand the meaning of
unfamiliar words and phrases and improving their language skills.

What is the application of Machine Translation?

Machine translation has many applications, including:
1.​ Cross-border communication: Machine translation allows people from
different countries to communicate with each other more easily,
 breaking down language barriers and facilitating international
cooperation.
2.​ Localization: Machine translation can be used to quickly and efficiently
translate websites, software, and other digital content into different
languages, making them more accessible to users around the world.
3.​ Business: Machine translation can be used by businesses to translate
documents, contracts, and other important materials, enabling them to
work with partners and customers from around the world.
4.​ Education: Machine translation can be used in education to help
students learn new languages and improve their language skills.
5.​ Government: Machine translation can be used by governments to
translate official documents and communications, improving
accessibility and transparency.