0% found this document useful (0 votes)
12 views5 pages

Social Media Data Analysis with EDA

The document outlines an experiment on Exploratory Data Analysis (EDA) and visualization of social media data, specifically using Twitter data. It discusses the use of the TextBlob library for sentiment analysis and the creation of Word Clouds for visual representation of data. The conclusion emphasizes the importance of EDA in cleaning and making data meaningful.

Uploaded by

mgade3012
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
12 views5 pages

Social Media Data Analysis with EDA

The document outlines an experiment on Exploratory Data Analysis (EDA) and visualization of social media data, specifically using Twitter data. It discusses the use of the TextBlob library for sentiment analysis and the creation of Word Clouds for visual representation of data. The conclusion emphasizes the importance of EDA in cleaning and making data meaningful.

Uploaded by

mgade3012
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

DEPARTMENT OF ARTIFICIAL INTELLIGENCE

& DATA SCIENCE

Subject: Social Media Analytics Course Code: CSDOL8023


Semester: VIII Course: AI & DS
Laboratory No: 302 Name of Subject Teacher: Prof. Gitanjali Korgaonkar
Name of Student: Meghana Gade Roll Id: VU2S2223002

Experiment no: 4
Aim: Exploratory Data Analysis and visualization of Social Media Data for business.

Theory:

Exploratory data analysis (EDA) is used by data scientists to analyze and investigate data sets and summarize their main
characteristics, often employing data visualization methods. It helps determine how best to manipulate data sources to get
the answers you need, making it easier for data scientists to discover patterns, spot anomalies, test a hypothesis, or check
assumptions.

TextBlob is a Python (2 and 3) library for processing textual data. It provides a simple API for diving into common
natural language processing (NLP) tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis,
classification, translation, and more.

Word Clouds came out to be a game-changer visualization technique for understanding and determining patterns and
evolving trends. Whether to discover the political agendas of aspiring election candidates of a country or to analyze the
customer reviews on the recently launched product, one can get a visual representation by plotting the Word Cloud.

1. Twitter Data Cleaning, Preprocessing and Exploratory Data Analysis:

df1=df1.drop_duplicates("renderedContent") [Link]
[Link] [Link].value_counts()
#Heat Map for missing Values [Link](figsize=(17, 5))
[Link]([Link](), cbar=True, yticklabels=False) [Link]("Column_Name", size=14, weight="bold")
[Link]("Places of missing values in column",size=17) [Link]()

import plotly.graph_objects as go
Top_Location_Of_tweet= df1['place'].value_counts().head (10) print(Top_Location_Of_tweet)

def get_subjectivity(text):
return TextBlob(text).[Link] def get_polarity(text):
return TextBlob(text).[Link]

df1['subjectivity']=df1[ 'renderedContent'].apply(get_subjectivity)
df1[ 'polarity' ]=df1[ 'renderedContent'].apply(get_polarity) [Link]()

2. Sentiment Analysis

df1['textblob_score'] =df1[ 'renderedContent'].apply(lambda x: TextBlob(x).[Link]) neutral_threshold=0.05


df1['textblob_sentiment']=df1[ 'textblob_score'].apply(lambda c:'positive' if c >= neutral_threshold
else ('Negative' if c <= -(neutral_threshold) else 'Neutral' ) )

textblob_df = df1[['renderedContent','textblob_sentiment','likeCount']] textblob_df


textblob_df["textblob_sentiment"].value_counts() textblob_df["textblob_sentiment"].value_counts().[Link](title,
Analysis',color='orange' , width=.4, figsize=(12,8),stacked = True) = 'Sentiment

df_positive=textblob_df[textblob_df['textblob_sentiment']=='positive' ]
df_very_positive=df_positive[df_positive['likeCount']>0] df_very_positive.head()
df_negative=textblob_df[textblob_df['textblob_sentiment']=='Negative' ] df_negative
df_neutral=textblob_df[textblob_df['textblob_sentiment']=='Neutral' ] df_neutral

3. Visualization by creating a Word Cloud

from wordcloud import WordCloud, STOPWORDS from PIL import Image


#Creating the text variable
positive_tw="".join(t for t in df_very_positive.renderedContent)
#Creating word_cloud with text as argument in .generate() rtpthod
word_cloud1=WordCloud (collocations=False, background_color='white'). generate (positive_tw)
# Display the generated Word Cloud [Link](word_cloud1,interpolation='bilinear) [Link]('off')
[Link]()

#Creating the text variable


negative_tw="".join (t for t in df_negative.rendered Content) #Creating word_cloud with text as argument [Link]()
rtpthod
word_cloud2=WordCloud(collocations=False,background_color='white').generate(negative_tw) # Display the generated
Word Cloud
[Link](word_cloud2,interpolation='bilinear') [Link]('off')
[Link]()

#Creating the text variable


neutral_tw="".join(tfor tindf_neutral.renderedContent)
#Creating word_cloud with text as argument in. generate() rtpthod
word_cloud2=WordCloud(collocations=False,background_color='white').generate(neutral_tw) # Display the generated Word Cloud
[Link](word_cloud2,interpolation='bilinear') [Link]('off')
[Link]()
Conclusion:

Exploratory data analysis helps to get data which is cleaned and meaningful. Twitter data is explored using textblob
package and visualization by using wordcloud package of python.

R1 R2 R3
DOP DOS Conduction File Record Viva Voice Total Signature
5 Marks 5 Marks 5 Marks 15 Marks
R1 R2 R3
DOP DOS Conduction File Record Viva Voice Total Signature
5 Marks 5 Marks 5 Marks 15 Marks

Common questions

Powered by AI

Sentiment analysis of social media data using TextBlob can categorize text into positive, negative, and neutral sentiments based on text polarity scores. This categorization allows analysts to gauge public opinion or emotional trends concerning specific topics or products. For example, by analyzing the polarity of tweets, businesses can intuitively understand customer satisfaction, immediate reactions to product launches, and overall brand sentiment. Tracking the frequency and distribution of each sentiment category provides actionable insights into consumer attitudes and their changes over time .

Word Clouds are important visualization tools in social media data analysis as they provide a graphic representation of word frequency in which more frequent words appear larger. This tool helps in understanding patterns and evolving trends by visually capturing the prominence of certain words in the data. For instance, Word Clouds can illustrate the main topics of customer reviews or reveal political agendas. By visually emphasizing the most common words, analysts can gain immediate insights into the key themes and discussions occurring in social media data .

Heat maps play a crucial role in visualizing and identifying issues such as missing values in social media data during EDA. By plotting heat maps, analysts can quickly pinpoint patterns of missing data across different columns, helping to understand the extent and nature of data quality issues. This visualization aids in deciding the appropriate cleaning strategies, such as imputing missing values or removing incomplete records, ultimately leading to enhanced data quality and reliability in further analysis .

A data scientist might choose to generate Word Clouds for Twitter data segmented by sentiment (positive, negative, and neutral) to qualitatively discern differences in discussions under each sentiment category. By examining Word Clouds, they can identify which topics, keywords, or phrases are most associated with each sentiment, thereby uncovering specific patterns or concerns linked to the sentiment. This can yield insights into the factors that drive positive or negative sentiment and may inform targeted strategies for improving public perception or addressing consumer complaints .

The primary objectives of Exploratory Data Analysis (EDA) when applied to social media data analytics are to analyze and investigate data sets to summarize their main characteristics and to utilize data visualization methods. This process helps data scientists discover patterns, spot anomalies, test hypotheses, and check assumptions, ultimately determining how to best manipulate data sources to extract relevant insights .

The application of sentiment polarity thresholds directly influences the classification results in sentiment analysis by defining the boundaries for each sentiment category. For example, a neutral threshold of 0.05 establishes that scores above this threshold are classified as positive, while scores below -0.05 are negative, and scores falling between these values are neutral. Adjusting thresholds can thus alter the proportion of classified sentiments, impacting the analysis' sensitivity to subtle polarity variations. Properly setting these thresholds is critical for achieving a balance between capturing nuanced sentiment and maintaining classification accuracy .

Preprocessing Twitter data, such as removing duplicates and handling missing values as part of EDA, enhances the reliability and accuracy of the analysis. Techniques like dropping duplicates ensure that the data set reflects unique, non-redundant information, which prevents bias in pattern recognition. Identifying and visualizing missing values, exemplified by heat maps, helps in understanding data quality, which is crucial for accurate insights. These steps ensure clean and meaningful data, improving the validity of further analytical processes, such as sentiment analysis .

Python libraries like TextBlob and WordCloud are highly effective in conducting EDA for social media analytics due to their specialized functionalities. TextBlob simplifies complex NLP tasks such as sentiment analysis and text classification, making it practical for analyzing large volumes of social media text data with efficiency. Combined with WordCloud, which visually represents word frequency, these tools enable comprehensive textual data exploration and trend visualization. These libraries collectively enhance the analytical process by providing both detailed and graphical insights swiftly and accurately, aiding in the extraction of actionable business intelligence .

Visualization techniques such as bar charts and heat maps complement TextBlob sentiment analysis by providing a structured visual representation of data distribution and quality, enhancing interpretability. Bar charts can display the frequency counts of sentiment categories, offering a clear picture of sentiment prevalence and shifts over time or across different data segments. Heat maps assist in identifying data quality issues like missing values, which could affect sentiment scores. Together, these visualizations provide a clearer, more comprehensive understanding of patterns and trends discovered through sentiment analysis, allowing for more informed decision-making .

The TextBlob library can be used for a variety of natural language processing tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis, classification, and translation. In the context of analyzing social media data, TextBlob can be employed to calculate the subjectivity and polarity of text, which are key metrics for determining sentiment. This is achieved by applying TextBlob's sentiment analysis functions to the 'renderedContent' of social media data to derive scores for subjectivity and polarity .

You might also like