DAV CSC601

Module 4. Text Analytics

Text mining

• Text mining is the process of removing

valuable data and complex patterns from

massive text datasets.

• The process of synthesizing information

through the examination of relationships,

trends, and rules amongst textual material is

known as text mining.

What Is Data
• Data is a set of facts such as descriptions, observations, and numbers used in

decision-making.

• Types of data

⚬ Structured data

⚬ Semi-structured Data

⚬ Unstructured Data
Structured data Semi-Structured data UnStructured data
• Tabular manner • Doesn’t consist of • Not organized in a pre-defined

• Rows and column Structured data manner

• Relation • Still has some • Does not have a pre-defined data

• Sql structure to it. model.

• A set of text-heavy
• Videos, audio, and binary data
files
 Text mining Text Analytics

Text mining involves the application of natural language

Text Analytics involves Statistical and linguistic methods.
processing and machine learning techniques.

To discover patterns, trends, and knowledge from large Text Analytics focuses on extracting meaningful
volumes of unstructured text. information, sentiments, and context from text

Text analytics emphasizes deriving actionable insights for

Text mining emphasizes uncovering hidden patterns,
decision-making.

Text mining exploring patterns. Text analytics providing interpretative context.

Text Mining Techniques
• Information Retrieval

• Information Extraction

• Natural Language Processing

• Data Mining
Information Retrieval
• In the process of Information retrieval, to process the available documents and the

text data into a structured form so, that apply different pattern recognition and

analytical processes.

• It is a process of extracting relevant and associated patterns according to a given set

of words or text documents.

• For this, processes like Tokenization of the document or the stemming process in

which we try to extract the base word.

[Link]
Information Extraction
It is a process of extracting meaningful words from documents.

• Feature Extraction –

⚬ To develop some new features from existing ones.

⚬ This objective can be achieved by parsing an existing feature or combining two or more

features based on some mathematical operation.

• Feature Selection –

⚬ To reduce the dimensionality of the dataset which is generally a common issue while

dealing with the text data by selecting a subset of features from the whole dataset.
[Link]
Natural Language Processing
Natural Language Processing includes tasks that are accomplished by using Machine Learning and

Deep Learning methodologies. It concerns the automatic processing and analysis of unstructured text

information.

• Named Entity Recognition (NER): Identifying and classifying named entities such as people,

organizations, and locations in text data.

• Sentiment Analysis: Identifying and extracting the sentiment (e.g. positive, negative, neutral) of

text data.

• Text Summarization: Creating a condensed version of a text document that captures the main

points.
Data Mining
• Large data sets are sorted through data mining in order to find patterns and connections that may

be used in data analysis to assist solve business challenges.

• Enterprises can forecast future trends and make more educated business decisions thanks to data

mining techniques and technologies.

The Seven Practice Areas of Text Analytics
• Search and information retrieval (IR)

• Document clustering

• Document classification

• Web mining

• Information extraction (IE)

• Natural language processing (NLP)

• Concept extraction
Five Questions to Target the Practice Areas
• Focus, especially the question of whether to emphasize search or information extraction – that is,

finding specific words and documents versus characterizing the entire data set;

• Granularity, the desired level of detail of focus;

• Available information, emphasizing whether there is enough information to provide a pattern

and look for that, or with less information, just look for anomalies;o999o

• Syntax versus semantics, which means deciding whether the analysis is more about what words

literally mean or what they connote in context; and

• Web information vs. traditional document-based text.

Why Is Text Mining Useful?
• The Explosion Of Text Stored In Electronic Format

• Text Data And Information

Applications and Use Cases for Text Mining
Five basic types of analytical text mining applications and use cases can be identified that

address most important text processing issues:

• Extracting “meaning” from unstructured text. Document clustering

• Automatic text categorization. Web mining

• Improving predictive accuracy in predictive modeling or unsupervised learning.

• Identifying specific or similar/relevant documents.

• Extracting specific information from the text (“entity extraction”).

Applications and Use Cases for Text Mining
Five basic types of analytical text mining applications and use cases can be identified that

address most important text processing issues:

• Extracting “meaning” from unstructured text.

• Automatic text categorization. Web mining

• Improving predictive accuracy in predictive modeling or unsupervised learning.

• Identifying specific or similar/relevant documents.

• Extracting specific information from the text (“entity extraction”).

Applications and Use Cases for Text Mining
Extracting “Meaning” from Unstructured Text

Summarizing Text
Text Analysis
• Text analytics is the process of transforming unstructured text in documents into

structured data which can be used for analysis.

• Text analysis is used to find patterns in the text and find meaning in them.

• It works by breaking phrases and sentences into components, and then evaluating the

meaning and roles using algorithms or software.

Text Analysis steps
The steps involved in analyzing an unstructured text document are :

• Language Identification

• Tokenization

• Sentence breaking

• Part of Speech tagging

• Chunking

• Syntax parsing

• Sentence chaining
Text Analysis steps
Language Identification
• The first step is to identify in which language the text is written.

• Since each language has its own rules of grammar, language identification is a major

process for every text analytics function.

• Its very important to know what language we will be dealing with.

Tokenization
• Tokenization is a technique that involves dividing a sentence or phrase into smaller units

known as tokens.

• Tokenization is the process of breaking down the sentence into small pieces. Tokens are the

words, numbers or punctuation in the sentence.

• In text analytics, tokens are usually words. For eg. a sentence of 15 words would have 15

tokens.
Types of Tokenization

• Word Tokenization: Word tokenization

divides the text into individual words.

• Sentence Tokenization:The text is

segmented into sentences during sentence

tokenization.
Types of Tokenization
• Subword Tokenization: Subword tokenization

entails breaking down words into smaller units,

which can be especially useful when dealing with

morphologically rich languages or rare words.

• Character Tokenization: This process divides the

text into individual characters. This can be

useful for modelling character-level language.

Need of Tokenization
• Effective Text Processing

• Feature extraction

• Language Modelling

• Information Retrieval

• Text Analysis

• Vocabulary Management

• Preprocessing Step
Types of Tokenization
1. Word Tokenization:

A piece of text is divided into individual words. For example,

• Sentence: “The quick brown fox jumps over the lazy dog”

• Tokens: “The”, “quick”, “brown”, “fox”, “jumps”, “over”, “the”, “lazy”, “dog”

2. Sentence Tokenization:

This technique involves breaking down a piece of text into individual sentences. For example,

• Paragraph: “The quick brown fox jumps over the lazy dog. It was a sunny day. The fox

was very happy.”

• Tokens: “The quick brown fox jumps over the lazy dog.”, “It was a sunny day.”, “The

fox was very happy.”

Types of Tokenization
3. N-gram Tokenization:

N-gram tokenization involves creating contiguous sequences of words from a piece of text.

• Paragraph: “The quick brown fox jumps over the lazy dog”

• Token bi-gram: “The quick”, “quick brown”, “brown fox”, “fox jumps”, “jumps over”,

“over the”, “the lazy”, “lazy dog”.

4. Stemming:

Stemming is a type of tokenization that involves reducing a word to its base form, or stem.

• For example, the stem of the word “jumps” is “jump”, and the stem of the word

“jumping” is also “jump”.

Types of Tokenization
5. Lemmatization:

Lemmatization is used in NLP because it can produce more meaningful and accurate tokens

than stemming.

• For example, the lemma of the verb “jumps” is “jump”, and the lemma of the noun

“jumps” is “jump”.

6. White space Tokenization:

This technique involves dividing a piece of text into tokens based on white space characters,

such as spaces, tabs, and newline characters. For example,

• Sentence: “The quick brown fox jumps over the lazy dog”

• Word tokens: “The”, “quick”, “brown”, “fox”, “jumps”, “over”, “the”, “lazy”, “dog”.
Types of Tokenization
7. Punctuation Tokenization:

This technique involves dividing a piece of text into tokens based on punctuation marks,

such as periods, commas, and exclamation points. For example,

• Sentence : “The quick brown fox jumps over the lazy dog!”

• Tokens: “The”, “quick”, “brown”, “fox”, “jumps”, “over”, “the”, “lazy”, “dog”, “!”.
Limitations of Tokenization
• Tokenization is unable to capture the meaning of the sentence hence, results in

ambiguity.

• In certain languages like Chinese, Japanese, Arabic, lack distinct spaces between

words. Hence, there is an absence of clear boundaries that complicates the process of

tokenization.

• Text may also include more than one word, for example email address, URLs and

special symbols, hence it is difficult to decide how to tokenize such elements.

Sentence Breaking
• Sentence boundary disambiguation (SBD), also known as sentence breaking, sentence

boundary detection, and sentence segmentation, is the problem in natural language

processing of deciding where sentences begin and end.

“ Dr. ---- does not indicate the end.

“ ”
Punctuation marks -----Shows start and end of sentences
Part of Speech Tagging
• Part-of-Speech (POS) tagging is a natural language processing technique that involves

assigning specific grammatical categories or labels (such as nouns, verbs, adjectives,

adverbs, pronouns, etc.)

• Part-of-Speech(POS) Tagging is the process of assigning different labels known as POS

tags to the words in a sentence that tells us about the part-of-speech of the word.

• Types of POS

⚬ Universal POS Tags

⚬ Detailed POS Tags

Part of Speech Tagging
• Part-of-Speech (POS) tagging is a natural language processing technique that involves

assigning specific grammatical categories or labels (such as nouns, verbs, adjectives,

adverbs, pronouns, etc.)

• Part-of-Speech(POS) Tagging is the process of assigning different labels known as POS

tags to the words in a sentence that tells us about the part-of-speech of the word.

• Types of POS

⚬ Universal POS Tags

⚬ Detailed POS Tags

List of Universal POS Tags
List of Universal POS Tags

These tags are the result of the division of universal POS tags into various tags, like NNS for

common plural nouns and NN for the singular common noun compared to NOUN for

common nouns in English. These tags are language-specific.

Advantages of POS Tagging

• Text Simplification

• Information Retrieval

• Named Entity Recognition

• Syntactic Parsing
Advantages of POS Tagging

• Text Simplification

• Information Retrieval

• Named Entity Recognition

• Syntactic Parsing
Chunking
• Chunking or light parsing refers to a range of sentence-breaking systems that fragment a

sentence into its component phrases (noun phrases, verb phrases, and so on).

• PoS tagging assigns parts of speech to tokens whereas Chunking assigns PoS-tagged

tokens to phrases.

• Chunking is defined as the process of natural language processing used to identify parts of

speech and short phrases present in a given sentence.

Chunking
For eg. take the sentence

The huge dog is going to run quickly across the bridge.

PoS tagging will identify dog and bridge as nouns and run as verb.

Chunking will return

• [The huge dog] as noun phrase

• [is going to run quickly] as verb phrase

• [across the bridge] as prepositional phrase

Syntax Parsing
• Syntax parsing is the process of determining how a sentence is formed.

• It is a critical step in sentiment analysis and other natural language processing

features.
Syntax Parsing
• Jack was fairing poorly in his exams until Mrs. Alice started teaching him.

Jack is negative, whereas Mrs. Alice is positive.

• Because Jack was fairing poorly in his exams, Mrs. Alice started teaching him.

Jack is still negative, but Mrs. Alice is now neutral.

• Jack was fairing poorly in his exams because Mrs. Alice started teaching him.

both Jack and Mrs. Alice are negative.

Sentence Chaining
• The final step is sentence chaining. Sentence chaining uses a technique to link individual sentences

using each sentence’s strength of association to an overall topic.

• For eg. take the following sentences:

⚬ I like hatchbacks.

⚬ Gabriella just bought a new SUV.

⚬ Audi launched a new sedan.

• Even if these sentences are not next to each other in a body of text, they are still connected through

the topics of hatchbacks->SUV->sedan. Sentence chaining lets us make these connections.

Benefits of Text Analytics
• Helps in understanding emerging customer trends, product performance, and service quality.

• Helps researchers to explore pre-existing literature and extracting what’s relevant to their study.

• Text analytic techniques help search engines to improve their performance, thereby providing fast

user experiences.

• Helps in making more data-driven decisions

• Refines user content recommendation systems by categorizing related content

• Boost Efficiency of working with Unstructured data

Use case Text Analysis
• Sports Trading

• Financial Trading

• Voice of the Customer (VOC)

• Fraud

• Manufacturing or Warranty Analysis

• Customer Service Routing

• Lead Generation

• TV Advertising & Audience Analysis

TF-IDF
• TF-IDF stands for Term Frequency Inverse Document Frequency of records.

• It can be defined as the calculation of how relevant a word in a series or corpus is to a text.

• TF-IDF is a numerical statistic that measures the importance of string representations such as words,

phrases and more in a corpus (document).

Corpus

• A corpus is a collection of authentic text or audio organized into datasets.

TF-IDF
• This is the mathematical equation to define TF IDF:

• t stands for term

• d stands for document

• D stands for set of documents

Term Frequency (TF)
The frequency of a particular term. The number of times a particular term is available in a corpus

can help us to measure the importance of that string.

• Raw count - How many times a word appears in the corpus.

• Boolean frequency - a Boolean data type is when there are two possible values - true/false, yes/no,

0/1. You can use 1 if the term occurs or 0 if the term does not occur

• Logarithmic scale - by using and displaying numerical data over a range of values.
Term Frequency (TF)

• t stands for term

• f stands for frequency

• d stands for document

Inverse Document Frequency (IDF)
How common a word is found in a corpus - or how uncommon a word is found in a corpus.

• IDF is important. Let’s take the English language for example, words such as “the”, “it”, “as”,

“or” which appear frequently in many types of documents.

• Inverse document frequency essentially minimizes the weight of frequency terms .

• t stands for term

• d stands for document

• D stands for set of documents

TF IDF
TF IDF
[Link]

[Link]

[Link]

Sentiment analysis

[Link]
Sentiment analysis
• Sentiment analysis is a popular task in natural language processing.

• The goal of sentiment analysis is to classify the text based on the mood or mentality expressed

in the text, which can be positive negative, or neutral.

• The goal that Sentiment mining tries to gain is to be analysed people’s opinions in a way that

can help businesses expand.

• It focuses not only on polarity (positive, negative & neutral) but also on emotions (happy, sad,

angry, etc.).

• It uses various Natural Language Processing algorithms such as Rule-based, Automatic, and
why sentiment analysis is important for business

• Sentiment analysis is a popular task in natural language processing.

• The goal of sentiment analysis is to classify the text based on the mood or mentality expressed

in the text, which can be positive negative, or neutral.

• The goal that Sentiment mining tries to gain is to be analysed people’s opinions in a way that

can help businesses expand.

• It focuses not only on polarity (positive, negative & neutral) but also on emotions (happy, sad,

angry, etc.).

• It uses various Natural Language Processing algorithms such as Rule-based, Automatic, and
why sentiment analysis is important for business

• Customer Feedback Analysis

• Brand Reputation Management

• Product Development and Innovation

• Competitor Analysis

• Marketing Campaign Effectiveness

Text Summarization

In Natural Language Processing, or NLP, Text Summarization refers to the process of using Deep

Learning and Machine Learning models to synthesize large bodies of texts into their most important

parts.

• Extractive Summarization

• Abstractive Summarization
Text Summarization
Extraction-based summarization
• In extraction-based summarization, a subset of words that represent the most important points

is pulled from a piece of text and combined to make a summary. Think of it as a

highlighter—which selects the main information from a source text.

• Extractive summarization aims at identifying the salient information that is then extracted and

grouped together to form a concise summary.

Extraction-based summarization

• A ranking algorithm is used, which assigns scores to each of the sentences in the text based on their

relevance to the overall meaning of the document.

• The most relevant sentences are then chosen to be included in the summary.

• There are various ways through which the ranking of sentences can be performed.

⚬ TF-IDF (term frequency-inverse document frequency)

⚬ Graph-based methods such as TextRank

⚬ Machine learning-based methods such as Support Vector Machines (SVM) and Random Forests.
Abstraction-based summarization
• Abstractive summary generation rewrites the entire document by building internal semantic

representation, and then a summary is created using natural language processing.

• In abstraction-based summarization, advanced deep learning techniques are applied to

paraphrase and shorten the original document, just like humans do.

• Think of it as a pen—which produces novel sentences that may not be part of the source

document.
Abstraction-based summarization

• This method uses advanced NLP techniques such as natural language generation (NLG) and

deep learning to understand the context and generate the summary.

Applications
Semantic Analysis