INTERNSHIP MINI-PROJECT REPORT
3. DATA CLEANING AND FEATURE ENGINEERING ON
TITANIC DATASET
Submitted By:
Kishore K
Department:
[Link] Artificial Intelligence and Data Science
College:
J.N.N Institute of Engineering
Internship Organization:
Edufyi Tech Solutions
TABLE OF CONTENTS
[Link] ............................................ 3
2. Introduction ..................................... 4
3. Dataset Description ……...................5
4. Methodology ......................................7
5. Results and Discussion ….................9
6. Conclusion ......................................10
2
1. ABSTRACT
Data cleaning and feature engineering are essential steps in the machine learning pipeline
because the quality of data directly affects model performance. Real-world datasets often
contain missing values, inconsistent formats, duplicate records, and irrelevant features that
must be addressed before model training. Proper preprocessing improves data quality and
enables machine learning algorithms to produce more accurate and reliable results.
The objective of this project is to perform data cleaning and feature engineering on the
Titanic dataset. The Titanic dataset contains passenger information such as age, gender,
passenger class, fare, cabin details, and survival status. The dataset was explored to identify
missing values and inconsistencies. Data cleaning techniques were applied to handle missing
values, remove unnecessary features, and ensure data quality.
Feature engineering techniques were then performed to create meaningful features, including
title extraction from passenger names and age categorization. Categorical features were
converted into numerical representations using encoding techniques, and feature scaling was
applied to normalize selected attributes. The resulting dataset became suitable for machine
learning applications and further analysis.
This project demonstrates the importance of data preprocessing in transforming raw data into
a structured and meaningful format that can be effectively used for predictive modeling and
data analysis.
3
2. INTRODUCTION
Data preprocessing is one of the most important stages in the machine learning lifecycle.
Before a machine learning model can be trained, the dataset must be cleaned and transformed
into a suitable format. Poor-quality data can lead to inaccurate predictions, reduced model
performance, and misleading conclusions. Therefore, data cleaning and feature engineering
play a critical role in ensuring the success of machine learning projects.
The Titanic dataset is one of the most widely used datasets for data science and machine
learning studies. It contains information about passengers who traveled on the RMS Titanic,
including demographic details, ticket information, passenger class, fare, and survival status.
The dataset presents several preprocessing challenges, such as missing values, categorical
variables, and irrelevant features, making it an ideal dataset for demonstrating data cleaning
and feature engineering techniques.
Data cleaning involves identifying and correcting issues within the dataset, such as missing
values, duplicate records, and inconsistent data formats. Feature engineering involves
creating new features or transforming existing features to improve the usefulness of the
dataset. Examples include extracting titles from passenger names, creating age groups,
converting categorical variables into numerical form, and scaling numerical features.
In this project, various preprocessing techniques are applied to the Titanic dataset to improve
data quality and prepare it for machine learning applications. The resulting dataset becomes
more structured, informative, and suitable for predictive modeling and data analysis.
4
3. DATASET DESCRIPTION
The dataset used in this project is the Titanic Dataset, one of the most popular datasets in
data science and machine learning. The dataset contains information about passengers who
traveled aboard the RMS Titanic, which sank during its maiden voyage in 1912. The
objective of the dataset is to analyze passenger characteristics and study factors that
influenced survival.
The dataset consists of 891 passenger records and 12 attributes. Each record represents a
passenger and includes demographic information, ticket details, travel class, fare, and
survival status. The dataset contains both numerical and categorical features, making it
suitable for demonstrating data cleaning and feature engineering techniques.
Dataset Information:
• Number of Records: 891
• Number of Attributes: 12
• Target Variable: Survived
• Type of Problem: Classification
The dataset contains the following attributes:
1. PassengerId – Unique identification number assigned to each passenger.
2. Survived – Indicates whether the passenger survived (1) or not (0).
3. Pclass – Passenger class (1 = First Class, 2 = Second Class, 3 = Third Class).
4. Name – Full name of the passenger.
5. Sex – Gender of the passenger.
6. Age – Age of the passenger.
7. SibSp – Number of siblings or spouses aboard.
8. Parch – Number of parents or children aboard.
9. Ticket – Ticket number.
[Link] – Fare paid by the passenger.
5
[Link] – Cabin number assigned to the passenger.
[Link] – Port of embarkation.
During the initial analysis, it was observed that some attributes contained missing values.
The Age column had missing entries, the Cabin column contained a large number of
missing values, and the Embarked column had a few missing records. These issues required
data cleaning before further analysis and feature engineering could be performed.
The Titanic dataset provides a valuable opportunity to apply preprocessing techniques such
as missing value handling, feature extraction, encoding, and normalization. These
transformations improve data quality and prepare the dataset for machine learning
applications.
Attribute Description
PassengerId Passenger ID
Survived Survival Status
Pclass Passenger Class
Name Passenger Name
Sex Gender
Age Passenger Age
SibSp Siblings/Spouses Aboard
Parch Parents/Children Aboard
Ticket Ticket Number
Fare Ticket Fare
Cabin Cabin Number
Embarked Port of Embarkation
Table 3.1 : Attributes and Description
6
4. METHODOLOGY
The methodology adopted in this project consists of several preprocessing steps aimed at
improving the quality of the Titanic dataset. These steps include data cleaning, handling
missing values, duplicate removal, feature engineering, categorical encoding, and feature
scaling. The processed dataset becomes suitable for machine learning and data analysis
applications.
4.1 Data Loading
The Titanic dataset was loaded into the Python environment using the Pandas library. Initial
exploration of the dataset was performed using functions such as head(), info(), and
isnull().sum() to understand the dataset structure and identify missing values.
4.2 Handling Missing Values
The dataset contained missing values in the Age, Cabin, and Embarked attributes. Since the
Cabin column contained a large number of missing values, it was removed from the dataset.
Missing values in the Age column were replaced using the median age value, while missing
values in the Embarked column were filled using the most frequent category (mode). After
processing, the dataset was checked again to ensure that no missing values remained.
Figure 4.1 : Missing Values After Data Cleaning
4.3 Duplicate Record Analysis
7
The dataset was examined for duplicate records using the duplicated() function. The analysis
showed that no duplicate records were present in the dataset. Therefore, no records needed
to be removed.
4.4 Feature Engineering
Feature engineering was performed to create additional meaningful attributes from existing
data. A new feature called Title was extracted from the Name column. Titles such as Mr,
Mrs, Miss, and Master provide useful demographic information and may contribute to
predictive analysis.
Another feature called AgeGroup was created by categorizing passengers into age-based
groups. This transformation helps simplify age-related analysis and enables machine
learning models to identify patterns more effectively.
4.5 Categorical Data Encoding
Machine learning algorithms require numerical input data. Therefore, categorical attributes
such as Sex, Embarked, Title, and AgeGroup were converted into numerical values using
label encoding techniques. This transformation ensured that all features could be processed
by machine learning algorithms.
4.6 Feature Scaling
The numerical attributes Age and Fare were standardized using the StandardScaler
technique. Feature scaling transforms the data so that it has a mean of zero and a standard
deviation of one. This process improves the performance of many machine learning
algorithms and ensures that features with larger values do not dominate the learning process.
4.7 Final Dataset Preparation
After completing data cleaning and feature engineering, the dataset contained only relevant
numerical features suitable for machine learning applications. The processed dataset was
free from missing values and properly formatted for further analysis and model development.
8
5. RESULTS AND DISCUSSION
The Titanic dataset was successfully cleaned and preprocessed. Missing values in the Age
and Embarked columns were handled appropriately, and the Cabin column was removed due
to a large number of missing entries. After preprocessing, the dataset contained no missing
values.
Feature engineering techniques were applied to create new attributes such as Title and
AgeGroup. Categorical features including Sex, Embarked, Title, and AgeGroup were
converted into numerical form using encoding techniques.
Furthermore, the Age and Fare attributes were standardized using feature scaling. The final
dataset became clean, structured, and suitable for machine learning applications. These
preprocessing steps improved data quality and prepared the dataset for further analysis and
predictive modeling.
Figure 5.1 : Standardized Numerical Features
9
6. CONCLUSION
In this project, various data cleaning and feature engineering techniques were successfully
applied to the Titanic dataset to improve its quality and suitability for machine learning
applications. The dataset was carefully analyzed to identify missing values, duplicate records,
and inconsistencies. Appropriate preprocessing methods were implemented to handle these
issues effectively.
Missing values in the Age and Embarked attributes were treated using statistical techniques,
while the Cabin column was removed due to the large number of missing entries. The dataset
was also examined for duplicate records, and the analysis confirmed that no duplicate
observations were present.
Feature engineering played an important role in enhancing the dataset. New features such as
Title and AgeGroup were created from existing attributes to provide additional information
that could be useful for analysis and predictive modeling. Furthermore, categorical features
were converted into numerical form through encoding techniques, and numerical attributes
were standardized using feature scaling methods.
The preprocessing steps transformed the raw Titanic dataset into a clean, structured, and
machine-learning-ready dataset. This project demonstrates the significance of data cleaning
and feature engineering in the data science workflow, as well-prepared data contributes
significantly to the performance and reliability of machine learning models. The processed
dataset can now be effectively used for classification, clustering, and other predictive
analytics tasks.
10