Social Media Moderation Using AI

MOTIVATION
Social media site one of the biggest type of online platforms that people use to
share and express their opinions by uploading content, a picture, or even
commenting on someone else’s. With the help of social media, people are now
easily able to stay informed of current events. People can easily and freely
upload anything on their accounts on social media or social networking sites
that other users can easily see. Taking advantage of this, many people misuse
such online platforms by uploading their offensive contents and sharing them
with a vast audience. User-generated content (UGC) enriches the internet's
content diversity but lacks traditional media's editorial oversight. This absence
of control allows some users to post content that can harm others, especially
children or vulnerable individuals. Examples include insensitive or cruel
content, terrorism promotion, and depictions of child abuse.

The traditional moderation of such posts requires that content to be flagged by

users of the platform as harmful; which is later inspected by a moderator
removed from the platform if post is found to be harmful. As the user-base of
the site increases such process become inefficient and are required to be
automated to keep up with increasing pool of user-generated content.

OBJECTIVE
The objective of this project is to create an AI-based system for automated
moderation of social media sites which can accurately moderate harmful
content. There is a broad range of content which is potentially harmful;
including but not limited to: hate speech, spam content, violent and extreme
content, graphic content, sexual content, cruel and insensitive material and
child abuse material. Some harmful content can be identified by analysing the
content alone, but other content requires an understanding of the context
around it to determine whether or not it is harmful. The system must analyse
the metadata of the post to conclude if the content falls within the policies
dictated by their individual site.
Scope of the project
• Content Moderation: This involves monitoring and managing user
generated content (UGC) to filter out inappropriate or harmful material
such as hate speech, harassment, misinformation and illegal content.
• Compliance: Ensuring that content posted on social media platform
complies with relevant laws, regulation and platform polices, such as
data protection laws and advertising standards.
• User Safety: Implementing measures to protect users, especially
vulnerable groups such as children, from online threats such as cyber
bullying and predatory behaviour.
• Insights and Analytics: Utilizing moderation data and analytics to gain
insights into user behaviour, sentiment, and trends to inform marketing
and business strategies.
• Platform Integrity: Maintaining the integrity of the social media platform
by detecting and removing fake accounts, bots, and other forms of spam
or malicious activity.
• Crisis Management: Being prepared to respond to and manage crises
such as negative publicity, viral misinformation, or security breaches.

Project Description
The project focus on developing AI-based system that analyse text, image and
video posted on social media platform. The below figure show the workflow of
the automated content moderation system

Algorithm used To identify Harmful Content

Text
RNNs (Recurrent Neural Networks) are powerful for social media moderation
due to their ability to process and understand sequences of data.
Key Applications:
1. Inappropriate Content Detection: Identifies hate speech, cyberbullying,
and explicit content by understanding context and language nuances.
2. Spam Detection: Analyzes patterns in messages to filter out spam.
3. Sentiment Analysis: Gauges sentiment to flag negative or harmful posts.
 4. Behavior Analysis: Monitors user actions over time to predict guideline
violations.
5. Multilingual Support: Handles different languages and dialects for global
platforms.
Advantages:
 Contextual Understanding: Effective for nuanced language processing.
 Temporal Analysis: Analyzes data changes over time.
 Variable Length Input: Suitable for diverse post lengths.
Challenges:
 Training Data: Needs large, diverse datasets.
 Computationally Intensive: Requires significant resources.
 Long Sequence Handling: Standard RNNs struggle, but LSTMs and GRUs
help.
RNNs are essential for automating social media moderation, offering
sophisticated tools for content and sentiment analysis, spam detection, and
behavior prediction, though they demand careful implementation and
resources.

IMAGE
Image recognition using convolutional neural networks (CNNs) involves using
deep learning algorithms to identify and classify objects in images. CNNs are
particularly effective for this task because they can automatically learn features
from the images, reducing the need for manual feature extraction.
Here is how we will adapt algorithm to detect harmful content.
1. Data Collection: Gather a dataset of images containing both
inappropriate/harmful images and normal images. These images should
be labeled accordingly.
2. Data Preprocessing: Preprocess the images to ensure they are in a
suitable format for input into the CNN. This may include resizing,
normalization, and augmentation.
 3. Model Architecture: Design a CNN architecture suitable for image
classification. You may use existing architectures like VGG, ResNet, or
design your own based on the complexity of the task and available
resources.
4. Training: Train the CNN using the pre-processed image dataset. During
training, the CNN learns to classify images as either
inappropriate/harmful or normal.
5. Validation: Validate the trained CNN using a separate dataset to ensure
it generalizes well to new, unseen images.
6. Testing: Test the CNN on a separate test dataset to evaluate its
performance in detecting inappropriate/harmful images.
7. Deployment: Once the CNN has been trained, validated, and tested, it
can be deployed to detect and potentially remove inappropriate and
harmful images on social media platforms. Detected images can be
flagged for review by human moderators or automatically removed
based on the platform's policies.

The two main point of this work flow are

• Pre-Moderation: This is when the content is moderate prior

to publication.
 • Post Moderation: When content is moderated after it has
been published and it has been flagged by other users or
automated processes as potentially harmful, or which was
removed previously but requires a second review upon
appeal.
1. Pre-Moderation by AI
• Here the content uploaded by user such as Text, Image, Audio, Video and
Metadata are analysed and flagged as not harmful, uncertain or harmful.
• Not harmful: This implies that the post has been evaluated by the AI and
it adheres to the polices of the organization
This content is made public and can be potentially viewed by all users of
the said platform.
• Uncertain: All AI system have a margin of error and its possible that the
it is unable to determine whether the content violates any policy of the
platform or not. Such posts are flagged as uncertain and are switched
over to the post moderation step.
• Harmful: The post flagged as harmful implies that it infringes some of
the policy of the site and hence that content is removed.
• Post Moderation
2. Post Moderation using human effort with assistance of AI
• Post moderation is done by human only if the following situation occurs.
• Flagged by user as harmful: The content which was public is branded as
harmful by some users of the website hence the post need human
inspection to determine it as harmful or not.
• User appeal: The content which was removed due to being harmful but
the poster of the content opposes the notion and appeals for a re-
evaluation.
• Uncertain: The content which was flagged uncertain due to limitation
AI-system to classify the post.
• The content in the above cases are evaluated by human effort with
assistance of AI. As the pool of content to be examined reduces
considerably AI can more thoroughly inspect the content
• After this step the content is flagged harmful or not harmful and are
made public or removed accordingly.
3. Training Feedback
• Data collected after post moderation is send as feedback to training data
to make the pre moderation step more accurate.

Hardware and Software Requirements

Hardware
 Laptop

Software
• PyCharm
• Docker
• Matlab
• React
• sentenceBERT and LLMs

Website
• Tensor flow
• Kaggle
• GitHub
• Hugging Face

Testing
The working and the efficiency of the AI based moderation system will be
tested using a clone of a social media site such Instagram or Twitter.

Reference
• Use of AI in Online Content Moderation 2019 Report Produced on behalf
of Ofcom by Cambridge Consultants
• Using Social Media in Community Based Protection By UNHCR
Guide
- Cassandra Maam
Group Members detail
Sr. No. Roll No Name Email Contact No.
1. 22CE61 Sharvil sharvilnaik170703@[Link] 7083581204
Madhav
Naik
2. 22CE80D Aaryan bhendeaaryan@[Link] 9405671171
Viswamber
Bhende