Syntax Analysis
POS Tagging (Parts-of-Speech Tagging)
Definition
POS Tagging is the process of assigning a grammatical label (part of speech) to each word in
a sentence, such as noun, verb, adjective, adverb, etc. This is done after the sentence is
tokenized into individual words.
Detailed Explanation
1. Process –
o First, the sentence is broken down into tokens (individual words).
o Then, each token is assigned a POS tag that tells its grammatical role in the
sentence.
2. Simple meaning –
POS tagging is like telling the function of each word in a sentence.
Example:
o “She (Pronoun) runs (Verb) fast (Adverb).”
3. Classification task –
It can be thought of as a classification problem in Natural Language Processing
(NLP), where each token is classified into one of the predefined POS categories.
4. Parts of Speech include –
o Noun – Names of people, places, or things
o Verb – Actions or states
o Adjective – Words that describe nouns
o Adverb – Words that modify verbs/adjectives
o Pronoun, Conjunction, Preposition, etc.
5. Approaches for POS Tagging –
o Rule-based tagging – Uses hand-written linguistic rules.
o Stochastic tagging – Uses probability models (e.g., Hidden Markov Models,
n-grams).
o Transformation-based tagging (TBL) – Learns patterns and corrects
mistakes using transformation rules.
Challenges in POS Tagging
1. Multiple tags for a word –
Many words can have more than one possible part of speech depending on the
sentence.
Example:
o “Book a ticket” → Book is a verb.
o “Read this book” → Book is a noun.
2. Unknown words (Out-of-Vocabulary) –
New words, slang, or domain-specific terms that are not in the training data can be
difficult to tag correctly.
3. Ambiguity –
A sentence may have more than one correct interpretation.
Example:
o “I saw the man with the telescope.” – Who has the telescope? The man or “I”?
4. Context dependency –
The POS of a word can change depending on the surrounding words.
Approaches to Perform POS Tagging
1. Rule-Based POS Tagging
Definition:
Uses a set of hand-written linguistic rules to determine the correct POS tag for each word.
How it works:
• Input: A string (sentence).
• Tag Dictionary / Lexicon: Assigns possible POS tags for each word.
• Handwritten Rules: Analyze linguistic features (suffixes, prefixes, capitalization,
word position).
• If a word has more than one possible tag, rules are applied to remove ambiguity.
• Output: Each word is assigned a single POS tag.
Example:
• “Book the ticket” → Book = Verb.
• “Read the book” → Book = Noun.
2. Stochastic POS Tagging
Definition:
Uses probability and frequency to decide the most likely POS tag for a word in a given
context.
How it works:
• Based on statistical models like Hidden Markov Models (HMMs).
• Word Frequency Measurement (Unigram): Counts how often a word appears with
each tag in a dataset and assigns the most frequent one.
• Tag Sequence Probability (Bigram/Trigram): Calculates the likelihood of a
sequence of tags for the entire sentence and chooses the most probable sequence.
• Selects the tag sequence with the highest probability.
Example:
If “run” appears as a verb 80% of the time and as a noun 20% of the time in training data, it is
more likely to be tagged as a verb when context is unclear.
3. Transformation-Based Tagging (TBL)
Definition:
Also called Brill Tagging, it is a hybrid approach that starts with an initial tagging (rule-based
or statistical) and then applies transformation rules to correct errors.
How it works:
• Initially tags words using a simple method (rules or frequency-based).
• Applies learned transformation rules to fix incorrect tags.
• Allows linguistic knowledge to be stored in a readable, rule-like form.
• Rules transform one state to another to improve accuracy.
Example:
A transformation rule might say:
“If a word is tagged as a noun but follows a determiner and a verb, change it to a verb.”
Potential Problems in CFG (Context-Free Grammar)
1. Agreement Problem
Definition:
Agreement refers to the grammatical relationship where certain parts of a sentence must
match in number, gender, or person.
CFG cannot easily enforce these agreement rules.
Explanation:
In English, a subject and verb must agree in number (singular/plural) and person.
However, in CFG, we usually have rules like:
S → NP VP
This rule allows any noun phrase (NP) to combine with any verb phrase (VP) —
even when they do not agree.
Example:
• Correct: He walks. (singular subject + singular verb)
• Incorrect: He walk. (singular subject + plural verb)
CFG will generate both sentences because it doesn’t check for agreement.
Conclusion:
CFG needs additional features or constraints to enforce agreement, which makes the
grammar very complex.
2. Subcategorization Problem
Definition:
Subcategorization means that certain verbs require specific types or numbers of
complements (objects).
CFG cannot easily restrict which kinds of complements follow which verbs.
Explanation:
Different verbs demand different sentence structures:
• Some verbs need no object,
• Some need one object,
• Some need two objects,
• Some need a clause.
But a simple CFG rule like:
VP → V NP
treats all verbs as if they take the same kind of complement.
Examples:
Verb Type Example Explanation
Intransitive He sleeps. No object allowed
Transitive He ate an apple. Needs one object
Ditransitive He gave her a gift. Needs two objects
Verb Type Example Explanation
Clause-taking He thinks that she is smart. Needs a clause
CFG cannot ensure that “sleep” doesn’t take an object or that “give” must have two.
Conclusion:
CFG lacks the ability to handle verb–complement restrictions, leading to grammatically
incorrect or incomplete sentences.
3. Movement Problem
Definition:
Movement refers to the phenomenon where words or phrases move from their normal
position to another place in the sentence (common in questions and relative clauses).
CFG cannot easily represent these movements.
Explanation:
In English, question formation and relative clauses involve moving parts of a sentence to the
front.
For example, in a question, the wh-word (like what, who, where) moves to the beginning.
Example:
• Base sentence: You will eat what.
• Question form: What will you eat?
Here, the object “what” moves from the end to the beginning of the sentence.
CFG uses fixed rules like:
S → NP VP
VP → V NP
So, it cannot easily describe such rearrangements where parts of the sentence move across
positions.
Another Example (Relative Clause):
• The book that I read yesterday was interesting.
→ “that I read yesterday” modifies “book,” but the noun “book” is missing from the
embedded clause.
CFG cannot naturally represent this gap or movement.