A REPORT ON
NEWS CATEGORY CLASSIFIER
MINOR PROJECT REPORT
SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENT FOR THE
AWARD OF THE DEGREE OF
BACHELOR OF TECHNOLOGY
(Computer Science & Technology)
SUBMITTED BY
(Arnav Mahajan, 2336768, CSE 5B1)
UNDER THE GUIDANCE OF
Dr. Renu Puri
Associate Professor
Department of Computer Science & Engineering
Chandigarh Engineering College –CGC Landran
Kharar-Banur Highway, Sector-112 Greater Mohali,
Punjab – 140307 (INDIA)
(Approved by AICTE, New Delhi and Affiliated to IKGPTU, Jalandhar)
October 2025
1
CANDIDATE'S DECLARATION
We hereby certify that the work, which is being presented in the project report, entitled “News
Category Classifier” by “Arnav Mahajan” in partial fulfillment of requirements for the award of
degree of [Link]. (CSE) submitted in the Department of CSE at Chandigarh Engineering College
under IKJ PUNJAB TECHNICAL UNIVERSITY, JALANDHAR is an authentic r e c o r d o f
my own work carried out during a period from July 2024 to October 2024 under the
supervision of _____________--.
Arnav Mahajan
Name & Signature of the Student
This is to certify that the above statement made by the candidate is correct to the best of our
knowledge.
The [Link] Viva –Voce Examination of Ishan Sharma and Jyoti has been held on and
accepted.
Dr. Renu Puri
Name & Sign of Project Guide
Associate Professor
2
ACKNOWLEDGEMENT
We would like to place on record my deep sense of gratitude to ________________ Dept. of
Computer Science & Engineering, CEC-CGC, Landran for his/her generous guidance, help and
useful suggestions.
We express my sincere gratitude to Dr. Sukhpreet Kaur, HoD, Department of CSE, CEC-CGC,
Landran & Dr. Manish Kumar, Professor, for their stimulating guidance, continuous
encouragement.
We also wish to extend my thanks to the online forums like Discord, Stack Exchange for their
insightful comments and constructive suggestions to improve the quality of this research work.
Arnav Mahajan
2336768
5th semester
CSE – 5B1
3
TABLE OF CONTENT
Content Page no
Objective 5
Technology Used 6
Design and Development 7-8
Process Flow 9-10
Screenshots of input and output 11-12
Code Snippets 13-18
Key Challenges and Solutions 19-20
Conclusion and Learning Outcome 21
4
OBJECTIVE
The primary objective of this project is to develop a machine learning-based text classification system that
can automatically categorize news articles into their respective topics, such as sports, politics, technology,
business, entertainment, and others, based on the content of the article.
The specific goals of the project include:
• Automate the classification process of textual news data using Natural Language Processing
(NLP), eliminating the need for manual sorting and labeling.
• Build a reliable and scalable ML model, specifically using the Random Forest Classifier, trained on
TF-IDF-transformed news content for multi-class prediction.
• Clean and preprocess raw text data using steps like lowercasing, stopword removal, punctuation
handling, and lemmatization to improve model accuracy.
• Visualize class distribution, accuracy, and model performance using metrics such as confusion
matrix, precision, recall, and F1-score to evaluate effectiveness.
• Design and deploy an intuitive Streamlit-based web interface, where users can:
o Upload datasets for training o Visualize categories and
statistics o Perform real-time predictions on custom news
inputs
• Enhance accessibility and usability of the model by providing a web-based app suitable for
journalists, media houses, and educators.
• Support balanced classification by considering techniques for handling skewed or imbalanced
categories, ensuring fair predictions across all labels.
This project not only demonstrates the application of NLP and ML in real-world scenarios but also emphasizes
usability by integrating the solution into a responsive web environment for seamless user interaction.
5
Technology used
Programming Language
• Python – The primary language used for data preprocessing, feature extraction, machine learning model
training, and web app development.
Libraries & Frameworks
• Pandas – For loading and manipulating the dataset (news text + category).
• NumPy – For numerical operations and data transformation.
• NLTK (Natural Language Toolkit) – For text preprocessing: tokenization, stopword removal,
lemmatization.
• Scikit-learn (sklearn) – For:
o TF-IDF vectorization o Train-test split o Random Forest
Classifier O Model evaluation (accuracy, F1-score,
confusion matrix)
• Matplotlib & Seaborn – For plotting class distributions and visualizing model performance metrics like
confusion matrix and bar charts.
Machine Learning Algorithm
• Random Forest Classifier – A robust and interpretable ensemble learning model used for accurate multi-
class classification of news articles.
Web Application Framework
• Streamlit – For building a responsive, interactive web interface that allows users to:
o Upload news datasets o Train and evaluate the model o Make
real-time predictions from custom news input o Visualize outputs
such as confusion matrix and accuracy scores ️ File Handling &
Data
• CSV Format – Used for dataset storage ([Link], [Link]), including article text and its category.
6
DESIGN AND DEVELOPMENT
The News Category Classifier was designed using a modular and layered architecture, ensuring that each
component—from data ingestion to model prediction and web deployment—was developed with clarity,
scalability, and usability in mind. The system follows a typical machine learning pipeline, enhanced with
Natural Language Processing (NLP) techniques and a Streamlit-based GUI for seamless user interaction.
🔹 1. Dataset Loading and Exploration
• The project begins with loading the news dataset ([Link]) which contains text content and their
associated categories.
• Pandas was used to explore the data, analyze class distribution, detect null values, and inspect text
samples.
🔹 2. Text Preprocessing (NLP Pipeline)
To ensure effective model training, the raw news text underwent comprehensive cleaning:
• Removal of punctuation, numbers, and special characters using regex.
• Conversion of text to lowercase for normalization.
• Tokenization of sentences and words using [Link].
• Removal of stopwords using [Link].
• Lemmatization using WordNetLemmatizer to reduce words to their base form.
• The cleaned text was stored in a new column called clean_text.
🔹 3. Feature Extraction
• The cleaned text was converted into numerical vectors using TF-IDF (Term Frequency–Inverse
Document Frequency) from sklearn.feature_extraction.text.
• This method captures the importance of words relative to documents, reducing noise and emphasizing
meaningful terms.
🔹 4. Label Encoding and Splitting
• The target categories were encoded into numerical labels using LabelEncoder.
• The dataset was split into training and test sets (80:20 ratio) using train_test_split() to evaluate
generalization.
7
🔹 5. Model Training
• The primary classifier used was Random Forest Classifier from [Link].
• The model was trained on the TF-IDF-transformed data.
• It was chosen for its robustness in handling high-dimensional data and multi-class classification.
🔹 6. Model Evaluation
• Model performance was assessed using:
o Accuracy score o Confusion matrix o Precision, recall, and
F1-score (via classification report)
• Visualization tools like Seaborn and Matplotlib were used to display results graphically.
🔹 7. Streamlit Web Interface
To enhance usability:
• A clean and responsive web interface was built using Streamlit.
• Key features include:
o File uploader for new datasets o
Dropdown to choose classifier o Text
box for manual news prediction o
Visual output for metrics and
predictions
• This made the app suitable for users without any programming background.
🔹 8. Screenshots and Testing
• Final screenshots were taken of the dataset preview, model output, confusion matrix, and live
predictions.
• The application was tested for various categories and input cases to validate accuracy and
responsiveness.
This design ensures the system is modular, extensible, and user-friendly, making it suitable for realworld use
in journalism, content filtering, and educational applications.
8
PROCESS FLOW
The end-to-end flow of the project is structured into the following major steps:
1. Dataset Import
a. Load the news dataset ([Link]) using Pandas.
b. Inspect text data and categories.
2. Text Preprocessing
a. Clean the text: remove special characters, numbers, and punctuation.
b. Convert text to lowercase.
c. Tokenize, remove stopwords, and apply lemmatization.
3. Feature Extraction
a. Apply TF-IDF Vectorization to convert text into numerical feature vectors.
4. Label Encoding
a. Encode news category labels into numeric values for ML compatibility.
5. Train-Test Split
a. Split the dataset into training and testing sets (e.g., 80:20 split).
6. Model Training
a. Use Random Forest Classifier to train the model on TF-IDF features.
7. Model Evaluation
a. Measure accuracy, precision, recall, and F1-score.
b. Plot confusion matrix to visualize model performance.
8. Web App Deployment
a. Build a Streamlit app to:
i. Upload CSV files
ii. Visualize data and results
iii. Input custom news and display predicted categories
9
FLOW CHART
Raw News Data
Text Cleaning & Lemmatization
TF-IDF Vectorization
Train/Test Split
Random Forest Classifier
Model Evaluation (Confusion Matrix, Accuracy)
Streamlit Web App (Input → Prediction)
10
Screenshots of Demo Input and Output
Input Screen (Homepage)
The homepage of the News Category Classifier is developed using Flask (Python backend) and a custom-
designed HTML & CSS frontend. It serves as the main entry point for users to interact with the classifier in a
user-friendly way.
• In this area we enter our news highlight we want to categorize
• then click on “Predict Category” option to find out the category the news belongs to
• Once we have entered the news (eg:- technological news), it will show us the category of technology
after the prediction.
11
12
Code Snippets
This block imports all essential libraries required for:
• Data handling (pandas, numpy)
• Text preprocessing (nltk)
• Machine learning algorithms (RandomForest, Logistic Regression, etc.)
• Evaluation metrics and plotting tools (matplotlib, seaborn)
• Model saving (pickle)
• Downloads necessary NLTK data files used for tokenization and lemmatization.
• Loads the [Link] and [Link] datasets into Pandas DataFrames for processing and training.
13
• Removes rows from the training and testing data that have null values in the text or label columns,
ensuring data cleanliness.
• Defines a function to lemmatize words (convert them to their base form), remove punctuation, and
lowercase the text. Applies it to both datasets.
• Converts the cleaned text data into numerical form using TF-IDF. This technique captures word
importance and ignores stopwords. It helps the model focus on meaningful terms in the news content.
• Transforms categorical text labels (e.g., ‘Politics’, ‘Sports’) into numeric values that can be used by
machine learning models.
14
Trains five different classifiers and predicts on the test data:
• Logistic Regression
• Random Forest
• Decision Tree
• Naive Bayes
• Gradient Boosting
This helps compare their performance and choose the best one.
15
• Generates a comparison table showing the performance (accuracy, precision, recall, F1-score) of each
model. This helps determine the most effective classifier.
• Uses bar plots and heatmaps to visually compare how well each model performs. These visualizations
are useful for presentations and interpretation.
• From the bar graph mentioned below it is clear that we are getting best accuracy of 91.3% or 0.913 from
logistic regression model among all the other models
16
• After we have done the backend work, we have created a website using Flask, HTML and CSS in [Link].
The functions of that [Link] performs are: -
• Loads the trained model and tools using pickle.
• Defines a Flask route to:
• Accept user input
• Preprocess the text
• Predict the category
• Display the result on the same page
17
18
Key Challenges and Solutions
Challenge 1: Textual Noise and Irrelevant Words
Description:
Raw news articles often contain HTML tags, punctuation, numbers, and stopwords that reduce model accuracy.
Solution:
Implemented comprehensive text preprocessing, including:
• Lowercasing
• Removing punctuation and digits
• Stopword removal using NLTK
• Lemmatization using WordNetLemmatizer
️Challenge 2: High Dimensionality of Text Data
Description:
News articles contain thousands of unique words, leading to sparse and high-dimensional vectors.
Solution:
Used TF-IDF Vectorization with parameters like max_df=0.7 and stop_words='english' to reduce noise and
dimensionality while retaining important terms.
️Challenge 3: Imbalanced Dataset
Description:
Certain categories like “Politics” or “Sports” may dominate, while others like “Technology” or “Business” are
underrepresented.
Solution:
Used weighted performance metrics (like weighted F1-score) during model evaluation to fairly assess
performance across all classes.
Challenge 4: Real-time Prediction Integration
Description:
Bridging the gap between the trained model and user-friendly interface was initially difficult.
Solution:
Used Flask web framework to build a lightweight app that:
• Accepts user input
19
• Processes and predicts category in real time
• Returns readable output on a clean HTML page
Challenge 5: Converting Model Output to Readable Labels
Description:
The model’s output was numeric (e.g., 0, 1, 2), not user-friendly.
Solution:
Used LabelEncoder.inverse_transform() to convert numeric labels back into their original category names (e.g.,
“Sports”, “Business”).
20
Conclusion and Learning Outcome
Conclusion
• The News Category Classifier project successfully demonstrates the use of Natural Language Processing
(NLP) and Machine Learning (ML) to automatically classify news articles into predefined categories
such as Politics, Sports, Business, and Technology. By combining efficient text preprocessing, TF-IDF
feature extraction, and the Random Forest classification model, the system achieved reliable and
interpretable results.
• Additionally, the integration of the trained model into a user-friendly Flask-based web application
enabled real-time predictions, making the solution accessible and interactive for end users. Despite
challenges like class imbalance and text variability, the project was able to deliver accurate and scalable
results with minimal latency.
Learning Outcomes
Through this project, the following technical and practical skills were developed:
• Understanding of text preprocessing techniques such as tokenization, stop word removal, and
lemmatization using NLTK.
• Proficiency with TF-IDF as a feature engineering method for converting text into meaningful numeric
representations.
• Hands-on experience with multiple ML models like Random Forest, Logistic Regression, Naive Bayes,
and Gradient Boosting.
• Ability to evaluate models using classification metrics like accuracy, precision, recall, F1-score, and
confusion matrix.
• Integration of ML models into web applications using Flask, along with HTML/CSS for frontend design.
• Deployment-ready workflow, including model serialization using Pickle and user interaction via forms
and route handling.
21