Group-17
Sentiment Analysis in Social Media
Dharani Kodati-11604652
Usha Bollepalli-11616926
Homakesh Thokala-11605244
Project Code
1/5
#group17-project
import nltk
[Link]('stopwords')
[Link]('punkt')
import pandas as pd
import [Link] as pltes
loading a reddit dataset which has 2 coliumns clean comment and category which
describing the clean comment contain the comments from user in english
category contain the numerical values -1,0,1 which indicate the positive,negative and
neutral reviews on the comments
our dataset has 37250 records which is a good number of records to perform sentiment
analysis on the comments.
import pandas as pd
data = pd.read_csv('/content/Reddit_Data.csv')
[Link](2)
#dropping the empety place or null values in the dataset
[Link](inplace=True)
# we are Replacing category labels for plotting negative, postive and neutral reviews on reddit dataset
data['category'] = data['category'].replace({-1: 'Negative', 0: 'Neutral', 1: 'Positive'})
# drwing the plot distribution of categories
[Link](figsize=(5, 4))
data['category'].value_counts().plot(kind='bar')
[Link]('Distribution of Categories')
[Link]('Category')
[Link]('Count')
[Link](rotation=0) # Rotating for better for readability
[Link]()
2/5
Below we are plotting the bar graph based on the review category and we can see
clearly more than 14000 and less than 1600 record fall into the categoryn of positive
and more than 12000 and less than 14000 and negative category approximately
nearly 8000.
import re
from [Link] import word_tokenize
from [Link] import stopwords
def clean_text(text):
text = [Link](r'[^\w\s]', '', text) # Removed punctuate
text = [Link]() # Converting-the text to lowerases
stop_words = set([Link]('english'))
tokens = word_tokenize(text)
text = [word for word in tokens if word not in stop_words] # Removing stopwords
return ' '.join(text)
data['clean_comment'] = data['clean_comment'].apply(clean_text)
The clean_text function, which gets textual data ready for tasks involving machine learning or further analysis. In order to concentrate on the text's
most important material, it methodically removes punctuation, changes all text to lowercase to maintain consistency, and gets rid of stopwords.
The NLTK library is used for tokenization and stopword removal, regular expressions are used to remove punctuation, and Python
list comprehensions are used for effective filtering. The filtered tokens are rejoined into a clean, processed text string to complete the function's
operation. When this technique is used on a DataFrame column, it greatly refines the dataset by removing unnecessary components.
from [Link] import LabelEncoder
#creating labels for categorical labels to integers.
label_encoder = LabelEncoder()
data['category'] = label_encoder.fit_transform(data['category'])
from sklearn.model_selection import train_test_split
#here we are distributed our dataset into test and train 20 and 80
X_D = data['clean_comment']
y_D = data['category']
X_train_D, X_test_D, y_train_D, y_test_D = train_test_split(X_D, y_D, test_size=0.2, random_state=42)
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from [Link] import Pipeline
model = Pipeline([
('tfidf', TfidfVectorizer()),
('clf', MultinomialNB())
])
[Link](X_train_D, y_train_D)
▸ Pipeline
▸ TfidfVectorizer
▸ MultinomialNB
3/5
from [Link] import accuracy_score, precision_score, recall_score, f1_score
y_pred_D = [Link](X_test_D)
accuracy = accuracy_score(y_test_D, y_pred_D)
precision = precision_score(y_test_D, y_pred_D, average='weighted')
recall = recall_score(y_test_D, y_pred_D, average='weighted')
f1 = f1_score(y_test_D, y_pred_D, average='weighted')
print("Accuracs are heres:", accuracy)
print("Precison are heres:", precision)
print("Recal are here:", recall)
print("F1score are here:", f1)
Accuracs are heres: 0.5449528936742934
Precison are heres: 0.7169212481262112
Recal are here: 0.5449528936742934
F1score are here: 0.4745762194316884
As seen from the data gotten through the analysis of the model by using clean_comment and category columns, one can be with certainty that
your text classification program satisfies its purposes but can be refined. The accuracy of 54.5% means that there's still space for improvement
since the model is just average in terms of correctly classifying comments into positive, negative and neutral posts with clean texts as input.
The high precision of approximately 71.7% suggests that the model in this case has a reasonable part of the categories guessed as right,
highlighting the model's strength as it has a fair vote of confidence when it eventually makes a choice. While the topic classification score at
54.5 % indicates the model has trouble covering all significant instances for each class, it holds the fact that the model misses quite a bit of
comments that should be correctly classified. The F1 Score (47.5%) represents a compromise situation in between precision and recall values
but still lacks in terms of high performance level across all subjects. Therefore, the model has problems in achieving more desirable results.
These results indicate the necessity of showing more finesse in how much certainty the model can faithfully predicts the categories precision and
how many instances it took to recall every category recall. For better accuracy we can use different model and algorithm which we discussing
below like SVM support vector machine.
# using SVM
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
# Load Data
data = pd.read_csv('/content/Reddit_Data.csv')
# Handle Missing Values
[Link](inplace=True)
# Text Vectorization
vectorizer = TfidfVectorizer()
X_D = vectorizer.fit_transform(data['clean_comment'])
y_D= data['category']
# Train-Test forSplit for svm
X_train_D, X_test_D, y_train_D, y_test_D = train_test_split(X_D, y_D, test_size=0.2, random_state=42)
from [Link] import SVC
# Chooseing svm Model for better performance
svm_model = SVC(kernel='linear')
# Train going on for Model below
svm_model.fit(X_train_D, y_train_D)
▾ SVC
SVC(kernel='linear')
from [Link] import precision_score, accuracy_score, f1_score, recall_score
# Predictions are doing below swe can see
y_pred_D = svm_model.predict(X_test_D)
# Calculating the - Metrics
4/5
accuracy = accuracy_score(y_test_D, y_pred_D)
precision = precision_score(y_test_D, y_pred_D, average='weighted')
recall = recall_score(y_test_D, y_pred_D, average='weighted')
f1 = f1_score(y_test_D, y_pred_D, average='weighted')
print("Accuracs are heres: ", accuracy)
print("Precison are heres : ", precision)
print("Recal are here : ", recall)
print("F1score are here :", f1)
Accuracs are heres: 0.8909825033647375
Precison are heres : 0.8903893759236574
Recal are here : 0.8909825033647375
F1score are here : 0.8886241573484381
when we compared these above 2 model naive bayes and svm we can see the difference in their performance clearly. Accuracy:
Naive Bayes: 54.5%
SVM: 89.1%
The SVM model gives very good results compared to the Naive Bayes classifier in terms of accuracy , and it is 35% increase in accuracy found in
between those. This implies that SVM is quite effective at the labeling of comments from your dataset even if there are variations in the
semantic context.
Precision:
Naive Bayes: 71.7%
SVM: 89.0%
Then SVM is again shown to be superior over Naive Bayes, which means that the mixture in the category of where it predicts has higher
reliability.
Recall:
Naive Bayes: 54.5%
SVM: 89.1%
The level of recall for SVM is very high implying that it is a much more effective technique in identifying all comments belonging to all different
categories and hence reducing the chances of inaccurately categorizing comments, a huge advantage for this algorithm.
F1 Score:
Naive Bayes: 47.5%
SVM: 88.9%
we can say from the above results that svm model for sentiment analysis task is performing well and more efficient than naive bayes model.
#ending by group17
5/5