Scribd
Upload
8 views3 pages

NLP Lab-5

The document explains lemmatization, a text preprocessing technique in Natural Language Processing that converts words to their base forms using vocabulary and grammar rules. It highlights the importance of lemmatization in reducing word variations, improving accuracy in NLP tasks, and cleaning text data, while also providing examples and applications. Additionally, it includes a Python program using NLTK to demonstrate lemmatization with sample outputs.

Uploaded by

vermaishank75
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views3 pages

NLP Lab-5

The document explains lemmatization, a text preprocessing technique in Natural Language Processing that converts words to their base forms using vocabulary and grammar rules. It highlights the importance of lemmatization in reducing word variations, improving accuracy in NLP tasks, and cleaning text data, while also providing examples and applications. Additionally, it includes a Python program using NLTK to demonstrate lemmatization with sample outputs.

Uploaded by

vermaishank75
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Experiment – 05

5. Write a Python program to perform lemmatization using NLTK.

5. What is Lemmatization?
• Lemmatization is a text preprocessing technique in Natural Language Processing that
converts a word into its base or dictionary form (lemma).
• It uses vocabulary and grammar rules (unlike stemming, which just cuts words).

Examples:
• running → run
• better → good
• studies → study

Why do we use Lemmatization?


We use lemmatization to:

1. Reduce word variations


→ "run", "running", "ran" → all become "run"
2. Improve accuracy in NLP tasks
→ Helps machines understand the actual meaning
3. Clean text data
→ Makes text consistent and easier to analyse.

Where is Lemmatization used?


Lemmatization is widely used in:

• Search Engines (Google search)


• Chatbots & Virtual Assistants
• Text Classification
• Sentiment Analysis
• Machine Translation
• Information Retrieval Systems

Example:
Sentence:
"The boys are running and playing games"

After lemmatization:
"The boy be run and play game"
Types of Lemmatizers
Type of Tool / Name Example Output
How it Works Accuracy
Lemmatizer Example Input (Lemma)
Uses grammar
Simple Rule
Rule-Based rules (remove playing play Medium
System
suffix)
WordNet
Dictionary- Uses dictionary
Lemmatizer better good High
Based lookup
(NLTK)
WordNet + POS Uses Part of meeting
POS-Based meet High
tagging Speech (verb)
Uses machine Very
ML-Based spaCy saw see
learning models High

Write a Python program to perform lemmatization using NLTK

Step-1
import nltk
[Link]('wordnet')

Step-2
from [Link] import WordNetLemmatizer
# Create lemmatizer
lemmatizer = WordNetLemmatizer()

print("Lemmatization Output:\n")

print("running ->", [Link]("running", 'v'))


print("better ->", [Link]("better", 'a'))
print("studies ->", [Link]("studies", 'n'))
print("playing ->", [Link]("playing", 'v'))

Lemmatization Output:
running -> run
better -> good
studies -> study
playing -> play

Another code
import nltk
[Link]('punkt_tab')
[Link]('wordnet')

#Lemmatization
from [Link] import word_tokenize
from [Link] import WordNetLemmatizer

text = "The cats are chasing mice and playing in the garden"
tokens = word_tokenize(text)
lemmatizer = WordNetLemmatizer()
lemmatized_text_list = [[Link](word) for word in tokens]
print("Original Text:", text)
print("Lemmatized Text:", lemmatized_text_list)

Original Text: The cats are chasing mice and playing in the garden
Lemmatized Text: ['The', 'cat', 'are', 'chasing', 'mouse', 'and', 'playing', 'in', 'the',
'garden']

You might also like