Scribd
Upload
7 views4 pages

Text Classification with LSTM Model

The document outlines a machine learning workflow for text classification using tweets, including data preprocessing, tokenization, and model training with a neural network. It employs techniques such as stemming, TF-IDF vectorization, and LSTM architecture for multi-class classification. The model's performance is evaluated and visualized through accuracy and loss plots, and the trained model is saved for future use.

Uploaded by

xewevo7586
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
7 views4 pages

Text Classification with LSTM Model

The document outlines a machine learning workflow for text classification using tweets, including data preprocessing, tokenization, and model training with a neural network. It employs techniques such as stemming, TF-IDF vectorization, and LSTM architecture for multi-class classification. The model's performance is evaluated and visualized through accuracy and loss plots, and the trained model is saved for future use.

Uploaded by

xewevo7586
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd

import os

import numpy as np
import pandas as pd
from [Link] import Tokenizer
from [Link] import pad_sequences
from [Link] import Sequential, load_model
from [Link] import Dense, Embedding, LSTM, Bidirectional,Flatten,
Dropout
from sklearn.model_selection import train_test_split
import re
from [Link] import stopwords
from [Link] import PorterStemmer
from sklearn.feature_extraction.text import TfidfVectorizer
import pickle

df=pd.read_csv("[Link]")

[Link]()

[Link]()

check_null = [Link]().sum()

check_null

[Link]

df["Tweet"].describe()

df["Category"].describe()

len(df["Tweet"])

for Tweet in range(len(df["Tweet"])):


df["Tweet"][Tweet]=[Link](r'<[^<>]+>', repl=" ",string=df["Tweet"][Tweet])
#remove html tags
df["Tweet"][Tweet]=[Link](r'[^a-zA-Z0-9\s]', repl=" ",string=df["Tweet"]
[Tweet]) #remove special characters/whitespaces

[Link]()

df["Tweet"][1]

port_stem = PorterStemmer()
def stemming(content):
#replace any non-alphabetic characters in the content variable with a space
character
stemmed_content= [Link]('[^a-zA-Z]',' ',content)
#Convert all words into lower case letters
stemmed_content = stemmed_content.lower()
# Split the words into list
stemmed_content = stemmed_content.split()
#generate a list of stemmed words from stemmed_content, excluding any stop
words from the list
stemmed_content = [port_stem.stem(word) for word in stemmed_content if not word
in [Link]('english')]
#Join the elements from the list 'stemmed_content' into a single string
separated by spaces
stemmed_content = " ".join(stemmed_content)
return stemmed_content

df["Tweet"]= df["Tweet"].apply(stemming)
df["Tweet"]

tokenizer = Tokenizer(num_words=5000) # unique words limit set to 5000

tokenizer.fit_on_texts(df['Tweet'])

X = tokenizer.texts_to_sequences(df['Tweet'])

X[0]

len(X[0])

# padding so that all reviews will be of length 500


X = pad_sequences(X,maxlen=500)

X[0]

len(X[0])

# Convert tokenized and stemmed sequences back to text format


documents = []
for sequence in X:
text = " ".join([str(token) for token in sequence if token != 0])
[Link](text)

# Create an instance of TfidfVectorizer


vectorizer = TfidfVectorizer()

# Compute TF-IDF scores on the entire dataset


x = vectorizer.fit_transform(documents)

print(x)

Y = df['Category']

X_train, X_test, Y_train, Y_test = train_test_split(x, Y, test_size=0.2)

X_train.shape

pickle_out=open('[Link]',"wb")
[Link](tokenizer,pickle_out)
pickle_out.close()

y_train=pd.get_dummies(Y_train)
y_test=pd.get_dummies(Y_test)

y_train

vocab_size = len(tokenizer.word_index) + 1 # +1 is necessary for embedding method

vocab_size

print(y_train.shape)
print(y_test.shape)
y_train.head()

from [Link] import Sequential


from [Link] import Dense
from [Link] import EarlyStopping
from [Link] import Reshape, Bidirectional, LSTM

# Convert sparse matrix to dense NumPy array


X_train_dense = X_train.toarray()

# Convert DataFrame to NumPy array


y_train_np = y_train.to_numpy()

# Define the model architecture with the correct output shape for multi-class
classification
num_classes = 4 # Number of actual number of classes in the data
timesteps = 64 # Set the desired number of timesteps

model = Sequential([
Dense(128, activation='relu', input_shape=(X_train_dense.shape[1],)),
Dense(64, activation='relu'),
Reshape((timesteps, -1)), # Reshape the output of the previous Dense layer to
(None, timesteps, features)
Bidirectional(LSTM(64)),
Dense(num_classes, activation='softmax') # Softmax activation for multi-class
classification
])

# Compile the model with categorical_crossentropy loss for multi-class


classification
[Link](optimizer='adam', loss='categorical_crossentropy',
metrics=['accuracy'])

[Link]()

earlyStopping = EarlyStopping(monitor='val_loss', mode='min', verbose=1,


patience=8)
# fitting the model with the updated architecture
modelTraining = [Link](X_train_dense, y_train_np,
batch_size=64,
epochs=15,
validation_data=(X_train_dense, y_train_np),
callbacks=[earlyStopping])

# Evaluate the model


score = [Link](X_train_dense, y_train_np, verbose=0)

print("Test_accuracy = ", score[1])

# Storing epoch values in variables


epochs = range(1, len([Link]['accuracy']) + 1)
accuracy = [Link]['accuracy']
loss = [Link]['loss']
val_accuracy = [Link]['val_accuracy']
val_loss = [Link]['val_loss']

import [Link] as plt

# Plotting
[Link](figsize=(12, 6))

[Link](1, 2, 1)
[Link](epochs, accuracy, 'r', label='Training Accuracy')
[Link](epochs, val_accuracy, 'b', label='Validation Accuracy')
[Link]('Training and Validation Accuracy')
[Link]('Epochs')
[Link]('Accuracy')
[Link]()

[Link](1, 2, 2)
[Link](epochs, loss, 'r', label='Training Loss')
[Link](epochs, val_loss, 'b', label='Validation Loss')
[Link]('Training and Validation Loss')
[Link]('Epochs')
[Link]('Loss')
[Link]()

plt.tight_layout()
[Link]()

[Link]('my')

Common questions

Powered by AI

Stemming reduces words to their base or root form, which helps in minimizing the feature space by treating related words as one. While removing HTML tags cleans up the data by eliminating non-text content, both steps enhance model performance by reducing noise and dimensionality, leading to more generalizable models .

The plots typically suggest convergence, where both training and validation accuracies improve while losses decrease, indicating effective learning. Disparities like increased validation loss or decreased accuracy suggest overfitting or issues in the data or architecture .

Early stopping monitors validation loss and halts training if no improvement is detected, preventing overfitting by ensuring the model does not spend unnecessary epochs learning noise from the training data, thereby maintaining model efficiency and performance .

Reshaping Dense layer outputs adjusts the data structure to one suitable for recurrent layers like LSTMs, which require input tensors of a specific shape (e.g., timesteps) to process sequences correctly; this ensures compatibility and efficient downstream processing .

TfidfVectorizer transforms text into a weighted numerical representation based on term frequency-inverse document frequency, highlighting terms that are more informative. This reduces the impact of frequent but less relevant words, making the feature set more discriminative compared to simple tokenization, which might treat all terms equally .

Converting a sparse matrix to a dense format can simplify model input handling but may increase memory consumption significantly, impacting training scalability. It provides computational ease at the cost of higher memory overhead, necessitating efficient resource management .

Padding sequences ensures uniform input dimensions required by models that process batches of data, like LSTMs. This prevents dimensionality issues, allowing consistent processing and comparison of sequences regardless of their original lengths .

Dropout layers randomly deactivate neurons during training, which prevents overfitting by forcing the model to learn robust features rather than relying on specific patterns in the training data, leading to improved generalization on unseen data .

The embedding layer maps high-dimensional sparse data into a lower-dimensional dense representation, capturing semantic relationships between words by placing similar words closer in the vector space, effectively providing meaningful input to subsequent layers like LSTMs .

Bidirectional LSTM processes sequences in both forward and reverse directions, capturing dependencies from past and future contexts, enhancing information retention and improving model understanding of context, which is crucial for sequential data like text .

You might also like