Scribd
Upload
5 views1 page

Assignment 4 Text Normalisation

The document outlines the steps for text normalization using a provided corpus related to Natural Language Processing (NLP). It includes links to various online tools for tasks such as sentence segmentation, tokenization, stopwords removal, lowercase conversion, stemming, and lemmatization. Additionally, it instructs on creating a document vector table, generating TFIDF values, and identifying words with the highest and lowest values.

Uploaded by

ambuj.sharma82
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
5 views1 page

Assignment 4 Text Normalisation

The document outlines the steps for text normalization using a provided corpus related to Natural Language Processing (NLP). It includes links to various online tools for tasks such as sentence segmentation, tokenization, stopwords removal, lowercase conversion, stemming, and lemmatization. Additionally, it instructs on creating a document vector table, generating TFIDF values, and identifying words with the highest and lowest values.

Uploaded by

ambuj.sharma82
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Text Normalisation

Q.1 Apply all the steps of text normalisation on the following corpus -
The Corpus
NLP - stands for “Natural Language Processing”. Using “NLP” technology, we can help a
machine understand human speech & spoken words. NLP enables chatbots to mimic
human conversation.

You can use the tools available online for these challenges. Link for each tool is given below:
1. Sentence Segmentation: [Link]
2. Tokenisation: [Link]

Note: Choose WordPunctTokenizer fron the result.


3. Stopwords removal: [Link]
4. Lowercase conversion: [Link]
5. Stemming: [Link]
6. Lemmatisation: [Link]
Show all the steps (screenshots) on A4 sized sheet and attach it in the practical file.

7. Bag of Words: Create a document vector table for all documents.


8. Generate TFIDF values for all the words.
9. Find the words having highest value.
10. Find the words having the least value.

You might also like