Scribd
Upload
3 views2 pages

Understanding Lemmatization in NLP

The document discusses lemmatization, a text preprocessing technique that produces a valid root word called a 'lemma' as opposed to a root stem from stemming. It highlights the use of the WordNetLemmatizer class from the NLTK library, which utilizes the morphy() function to find lemmas based on part of speech. Examples are provided to illustrate how lemmatization transforms various words into their base forms.

Uploaded by

Rajdip Ingale
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views2 pages

Understanding Lemmatization in NLP

The document discusses lemmatization, a text preprocessing technique that produces a valid root word called a 'lemma' as opposed to a root stem from stemming. It highlights the use of the WordNetLemmatizer class from the NLTK library, which utilizes the morphy() function to find lemmas based on part of speech. Examples are provided to illustrate how lemmatization transforms various words into their base forms.

Uploaded by

Rajdip Ingale
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

lemmatization-text-preprocessing

June 9, 2024

0.1 Wordnet Lemmatizer


Lemmatization technique is like stemming. The output we will get after lemmatization is called
‘lemma’, which is a root word rather than root stem, the output of stemming. After lemmatization,
we will be getting a valid word that means the same thing.
NLTK provides WordNetLemmatizer class which is a thin wrapper around the wordnet corpus.
This class uses morphy() function to the WordNet CorpusReader class to find a lemma. Let us
understand it with an example −
[19]: ## Q&A,chatbots,text summarization
from [Link] import WordNetLemmatizer

[2]: lemmatizer=WordNetLemmatizer()

[8]: '''
POS- Noun-n
verb-v
adjective-a
adverb-r
'''
[Link]("going",pos='v')

[8]: 'go'

[9]: words=["eating","eats","eaten","writing","writes","programming","programs","history","finally"

[14]: for word in words:


print(word+"---->"+[Link](word,pos='v'))

eating---->eat
eats---->eat
eaten---->eat
writing---->write
writes---->write
programming---->program
programs---->program
history---->history

1
finally---->finally
finalized---->finalize

[16]: [Link]("goes",pos='v')

[16]: 'go'

[18]: [Link]("fairly",pos='v'),[Link]("sportingly")

[18]: ('fairly', 'sportingly')

[ ]:

[ ]:

[ ]:

You might also like