Rudrani Chavarkar / B11 / DS Lab / Roll No.
ASSIGNMENT 6
Aim:
Implement Sentiment analysis using LSTM.
LO Mapped: LO4 - Develop real life applications using learning concepts.
Theory:
1. Steps in Text processing.
Text processing is a crucial step in natural language processing (NLP) that
involves transforming raw text into a structured format suitable for analysis. Here
are the key steps involved in text processing:
1. Text Cleaning: This initial step involves removing noise from the text data,
such as HTML tags, special characters, and irrelevant information. The goal
is to ensure that the text is clean and ready for further processing.
2. Tokenization: This process breaks down the cleaned text into smaller units
called tokens, which can be words or sentences. Tokenization is essential for
analyzing the structure of the text.
3. Lowercasing: Converting all text to lowercase helps maintain consistency,
ensuring that words like "Apple" and "apple" are treated as the same token.
4. Stop Words Removal: Stop words are common words (e.g., "is," "the,"
"and") that do not carry significant meaning in analysis. Removing these
words helps reduce dimensionality and focus on more meaningful terms.
5. Stemming and Lemmatization: These techniques reduce words to their base
or root forms. Stemming involves chopping off prefixes or suffixes, while
lemmatization considers the context and converts words to their dictionary
form.
6. Handling Contractions: Expanding contractions (e.g., changing "can't" to
"cannot") ensures uniformity in the text data.
7. Handling Emojis and Special Characters: In modern text data, especially
from social media, handling emojis and special characters is crucial for
maintaining context and sentiment.
Rudrani Chavarkar / B11 / DS Lab / Roll No. 2
8. Spell Correction: Correcting spelling errors in the text enhances the quality of
the data and reduces noise.
9. Normalization: This step involves converting variations of words to a
standard format (e.g., changing "smth" to "something") to ensure consistency
across the dataset.
[Link]-of-Speech Tagging: Identifying the grammatical roles of words in
sentences helps understand their context and relationships within the text.
2. Functioning of LSTM.
Long Short-Term Memory (LSTM) networks are a type of recurrent neural
network (RNN) designed to effectively learn long-term dependencies in sequential
data. They address the vanishing gradient problem that traditional RNNs face,
making them particularly useful for tasks involving time series, natural language
processing, and speech recognition.
Key Components of LSTM
1. Cell: The core memory unit that retains information over time.
2. Gates: LSTMs utilize three types of gates to control the flow of information:
● Input Gate: Determines which new information to add to the cell state.
● Forget Gate: Decides what information to discard from the cell state.
● Output Gate: Controls what information is output from the cell state.
Functioning of LSTM
● Input Reception: At each time step t, the LSTM receives an input vector xt and
the previous hidden state ht-1.
● Forget Gate Activation: The forget gate evaluates what information from the
previous cell state ct-1 should be discarded. It produces a value between 0 and
1 for each element in the cell state.
● Input Gate Activation: The input gate determines what new information will
be added to the cell state. It uses a sigmoid function to filter values and a tanh
function to create candidate values for updating the cell state.
● Cell State Update: The old cell state is updated by combining retained
information (from the forget gate) and new candidate values (from the input
gate).
● Output Generation: The output gate decides what part of the cell state will be
output as the hidden state ht. This involves filtering the cell state through a
sigmoid function and applying a tanh function to scale the values.
Rudrani Chavarkar / B11 / DS Lab / Roll No. 2
Advantages of LSTM
● Long-Term Dependencies: LSTMs excel at learning relationships over long
sequences, making them suitable for complex tasks like language
translation.
● Handling Sequential Data: They can process varying lengths of input
sequences, unlike traditional feedforward networks.
● Mitigation of Vanishing Gradient Problem: LSTMs are designed to maintain
gradients across many time steps, allowing for effective learning from long
sequences.
Conclusion:
Long Short-Term Memory (LSTM) networks are a significant advancement in the
field of recurrent neural networks, specifically designed to address the challenges
of learning long-term dependencies in sequential data. Overall, LSTMs have
proven to be a powerful tool in machine learning, enabling more accurate and
efficient processing of complex sequential data.