DATA PREPROCESSING
Data preprocessing in the data analysis is the important step where raw data is cleaned, transformed or
organized in a usual format
When raw data is incomplete, inconsistent, unstructured, the preprocessing data ensures the data
become reliable for analysis & decision making
The step of Data preprocessing below with python code:
Step 1: Read file ~ import to data frame and try to check some round to understand data frame
Step 2: Find the exploratory data analysis
[Link], [Link], [Link](),[Link]()
This step is the process of exploring datasets to summarize their main characteristics. This step explain
the size & structure data, the type of the variables (example: numeric, object,…), the distribution of
values (mean, median, outliers), the potential data quantity issues (example: missing values,
inconsistencies…)
2.1 [Link]
This return the number of rows & columns in the data set that help the analysis person to
understand the data.
The result below is the final data that is 3157 rows & 14 columns
2.2 [Link]
This return the type of the columns that analysis person can see all the types of columns to
replace to correct (if needed)
The result below is the types, after checking there have 1 columns is wrong that is
“Survey_Date”, that is object not the date time, so the analysis person should be change to date time
type
2.3 [Link]()
This step is the generated summary statistics for numeric columns
The results of below mean total of survey date is 3157, data survey from 01Jun2025 to
26Jun2025; total data of Age is 3157, from 22 to 65 years old, mean of age is 43.7; total data of Hours
per week is 3157, working hour is from 35 to 65 hours, the mean of hours per week is 49.9hours; total
data of work life balance & social insolation score are 3157, the score in this survey from 1 to 5
Step 3: Find missing value & replace
Finding the missing value is the important with data analysis, it check the data with empty, null or invalid
entries. If the data have the missing value, the result of analysis maybe wrong & wrong the making
decision. If we find the missing value in the data set, the data will be replace by mean / mode / … that is
belong to data understanding. If we don’t find any missing value, the process will be continue
The results of this data have 2 column with information is none “Mental Health Status” and “Physical
Health Issues”, but after checking sample of missing value, there isnot actual missing value, there are
just about the result of survey that mean No Issue with health impact. So, we choise the replace from
Null to “No Issue” for this null to analyst
Step 4: Find duplicate value & remove if there have the duplicated
In the data analysis, the duplicate value is important because if there have the duplicated data, the
result will be incorrect & the decision making is wrong too.
The result of this data is o duplicate value in this data, No need any action
Step 5: Find the columns with type Int64 for find the outliers data
In data analysis, an outlier is a data point that significantly deviates from other data points in a dataset.
It's an observation that doesn't conform to the expected pattern or distribution of the majority of the
data. Outliers can be significantly higher or lower in value compared to other data points and can
potentially distort statistical analysis and misrepresent the true nature of the data
For this data, I choice the method to find the outlier that is Interquartile range (IQR) that mean Values
outside the IQR (the middle 50% of the data) can be flagged as outliers
In data have 4 columns with Int64 are "Age","Hours_Per_Week", "Work_Life_Balance_Score",
"Social_Isolation_Score" that need to find the outliers
The result after run outliers that don’t have any outliers in this data, so no need do anymore in this step
Step 6: After preprocessing data done, we have to save a new file to do the next step