Scribd
Upload
25 views6 pages

Data Cleaning & ML Model Evaluation

Uploaded by

Puja Biswas
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
25 views6 pages

Data Cleaning & ML Model Evaluation

Uploaded by

Puja Biswas
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Title: Data Cleaning and Model Evaluation for Machine Learning Projects

Submitted by
Shreya Reddy
Lewis University

Course: Supervised Machine Learning

Instructor: Mahmood Al-Khassaweneh


Date: 10/17/2024
Abstract- In this report I present the how missing and inaccurate data were
outcome of programming projects in my managed, what machine learning models
master course for weeks 6 and 7. It was were used and how their performance was
designed to encompass tasks like cleaning assessed using standard metrics including
and reshaping raw data in a dataset, accuracy, precision, recall and F1-score.
applying several types of models drawn
from the field of machine learning to that II. DATA CLEANING PROCESS
dataset, and then assessing the A: Missing data: Its causes and
performance of the various models. management
Removing the noise was important in It should be noted that for this project, the
enhancing the quality of the models, and dataset has several rows with at least one
the following classification techniques missing or improperly filled value that
were used. This paper presents an could affect the operation of machine
overview of the data cleansing procedure, learning algorithms. Missing data was
the models applied in the analysis and the handled by employing multiple techniques,
assessment used. The outcome shows us including:
why it is crucial to clean the data and ● Imputation: Whenever the
select the correct model to enhance the excluded data was numerical, I
predictive power. applied mean and median
imputation to complete the missing
Keywords - data preprocessing, algorithm, data. In the case of categorical
classification models, metrics, accuracy data, the option of mode
imputation was considered.
I INTRODUCTION ● Removal of Incomplete Rows: As
The objective of the programming projects for some cases where many
in weeks 6 and 7 was to preprocess a given observations were missing, I
dataset, fit selected machine learning eliminated such rows to avoid
models on it, and compare the efficiency distorting the model analysis at
of the models to handle a classification hand.
task. In these weeks, I assumed a
consultant position and the goal was to B. Handling Outliers
explain why data preprocessing is crucial The dataset consisted of numeric fields
and how to select proper models to address and there were outlying values which
an actual problem. This report explains affected the results. The outliers for these
variables were corrected by using the model is, generally, utilised due to its
interquartile range (IQR) approach. They simplicity and ease of interpretation. In
are data points which fall below 1.5IQR or logistic regression, the possibility of a
above 75 IQR; they affect the training of given point x set in a certain class is
the models by leading it astray by estimated by mapping the data set to the
presenting patterns that do not apply. logistic function. However as we have
C. Feature Scaling and Normalisation seen the logistic regression model fared
As a result of this, feature scaling was used well as a baseline model for the dataset
to bring the values of the independent provided.
variables on the same platform that would
enable the machine learning models to
learn effectively from the data. This was
achieved by employing MinMaxScaler in
which all features are scaled to a range of B. Decision Trees
0 and 1. This is much needed in algorithms A Decision Tree classifier was also used.
such as SVM and KNN as these The idea of a decision tree is as follows,
algorithms work depending on the features the data set is divided into subsets on the
magnitude. basis of the features in the data set and it
D. Categorical Data Encoding forms a tree-like structure. Each node of
The assigned dataset included categorial the model is a decision rule with branches
features that were required to be encoded showing outcomes of the rule. Even
in order to feed them into the machine though this chosen model of building the
learning algorithms. One hot encoder was decision tree is reasonably easy to work
employed in order to encode the with in terms of interpretability and seems
categorical variables as binary vectors. to work well on a large number of case
This has been preferred over label studies it is sensitive to overfitting and
encoding because it does not bring any therefore needs to be regularly adjusted.
relationship between categories’ order in C. Random Forest
theStrings strings list. The problem of overfitting which I found
in the decision tree model is prevented by
III. MACHINE LEARNING MODELS using the Random Forest classifier which
A. Logistic Regression is an ensemble method built using decision
The first classification model used for the trees (Gong, et al, 2023). In classification
cleaned data was logistic regression. This
and regression, Random forests produce very accurate especially when we use the
several decision trees in the learning model on imbalanced datasets.
process and combine their classification or B. Precision ,Recall and F1 Score
regression. This model proved to be very Accuracy is defined as the ratio of totals
fast, especially when the appropriate that are correctly identified out of all totals
number of trees and the depth of each tree identified as such by a model (Lee, et al,
was decided. 2021). Sensitivity (or recall) quantifies the
D. Support Vector Machine number of actual positives assigned to the
I deployed a Support Vector Machine group of true positives among all existing
(SVM) , a kind of classification algorithm actual positives in the dataset.
that tries to find the hyperplane that creates F1-score is a combination of precision and
the biggest margin between the classes in recall ratio and it is recommended in cases
the feature space. SVM is more suitable of imbalanced data sets.
for use in high dimensions but this comes C. Confusion Matrix
with attendant problems, this includes the The confusion matrix helped to pry into
need to carefully select the values of the the performances of the models as it
regularisation term and the type of kernel showed the true positive counts, true
to be used. negative counts, false positive and false
negative counts. This matrix was
specifically useful in identifying where the
model was going wrong especially in
terms of the false positive and false
negative results.

IV. MODEL EVALUATION


To assess the effectiveness of each model,
I used several evaluation metrics:
A. Accuracy
Accuracy is the easiest metric which
predicts that out of all the instances on
each fold/round X%, were right. Accuracy
is a measure that gives overall
performance of the model, but it is not V. TABLE
solved the classification problem; the
Model Accu Preci recall F1
experience I got from this project is
racy sion Score
valuable in future machine learning
Logistic 85% 0.83 0.79 0.81 projects in dealing with real-world data
regressi chaos.
on

Decisio 80% 0.76 0.81 0.78


n Tree

Random 89% 0.85 0.84 0.84


forest

SVM 86% 0.83 0.82 0.82

TABLE 1
In the table the Random Forest classifier
outperformed other models in terms of
accuracy, precision, recall and F1 score.
The support vector machine also
performed well and slightly outperformed
logistic regression but falling short of
random forest’s performance.

VI. CONCLUSION
adding unto this week’s programming
projects from weeks 6 and 7 recapitulated
a crucial step of the machine learning
process: data cleaning and preparation.
Despite not being able handle missing
data, addressing outliers, and scaling
features, further enhanced the outcomes of
the model. In their application, Random
Forest was identified as the most effective,
with Support Vector Machines close
behind. These models fitting appropriately
VII. REFERENCES Computer Trends and Technology, 71(9),
Dataset 16-27.
Link:-[Link] [6]. Gong, Y., Liu, G., Xue, Y., Li, R., &
02/dry+bean+dataset
Meng, L. (2023). A survey on dataset
[1]. Lee, G. Y., Alzamil, L., Doskenov, B.,
quality in machine learning. Information
& Termehchy, A. (2021). A survey on data
and Software Technology, 162, 107268.
cleaning methods for improved machine
learning model performance. arXiv
preprint arXiv:2109.07127.
[2]. Lwakatare, L. E., Rånge, E., Crnkovic,
I., & Bosch, J. (2021, May). On the
experiences of adopting automated data
validation in an industrial machine
learning project. In 2021 IEEE/ACM 43rd
International Conference on Software
Engineering: Software Engineering in
Practice (ICSE-SEIP) (pp. 248-257).
IEEE.
[3]. Guha, S., Khan, F. A., Stoyanovich, J.,
& Schelter, S. (2024). Automated data
cleaning can hurt fairness in machine
learning-based decision making. IEEE
Transactions on Knowledge and Data
Engineering.
[4]. Habib, M., & Okayli, M. (2024).
Evaluating the sensitivity of machine
learning models to data preprocessing
technique in concrete compressive strength
estimation. Arabian Journal for Science
and Engineering, 1-19.
[5]. Rangineni, S. (2023). An analysis of
data quality requirements for machine
learning development pipelines
frameworks. International Journal of

Common questions

Powered by AI

Key steps in the data cleaning process include managing missing data through imputation or removal, handling outliers using techniques like the Interquartile Range (IQR), and scaling features to ensure consistency in data magnitudes. These steps are critical as they directly affect the model's ability to learn accurate patterns, thus improving model performance and predictive power .

The F1-score is significant in model evaluation for imbalanced datasets because it balances precision and recall, capturing both the model's ability to correctly identify positive instances and its completeness in capturing all positive occurrences. In imbalanced datasets where one class may outweigh others substantially, F1-score provides a better sense of model performance than accuracy alone, as it penalizes both false positives and false negatives .

Feature scaling and normalization, such as MinMaxScaler, adjust the data to a consistent scale, crucial for models like SVM and KNN since these algorithms rely on the distance between data points. Without uniform feature scaling, features with larger ranges can disproportionately affect the model's learning process, leading to suboptimal performance .

Using accuracy as the sole metric is limited in imbalanced datasets because it can be misleading. Accuracy measures the overall correctness of a model but does not consider the distribution of different classes. In imbalanced datasets, a model might trivially achieve high accuracy by predicting the majority class consistently, while failing to capture the minority class effectively, necessitating additional metrics such as precision, recall, and F1-score to evaluate performance comprehensively .

Handling outliers using the Interquartile Range (IQR) method mitigates their impact on a model by removing data points that lie significantly above or below the typical data range. This ensures that the model is not misled by extreme values that could distort the learning process, ultimately leading to more stable and accurate model predictions .

The choice of machine learning model affects effectiveness in handling classification tasks based on model characteristics such as simplicity, interpretability, and ability to handle overfitting. In the study, the Random Forest model outperformed others due to its ensemble nature, which prevents overfitting, achieving higher accuracy, precision, recall, and F1 score compared to models like Decision Trees, Logistic Regression, and Support Vector Machines .

Data cleaning and preparation set the foundation for successful machine learning projects by ensuring data is accurate, representative, and suitable for modeling. These processes address real-world data chaos through meticulous handling of missing values, outliers, and feature scaling, which enhance model reliability and predictive accuracy. They form crucial steps that significantly influence the overall success and generalization capability of machine learning models in future projects .

One-hot encoding is advantageous over label encoding as it converts categorical variables into binary vectors without introducing ordinal relationships between categories. This avoids misleading the model into interpreting the encoded categories as having inherent order, which is critical when feeding the data into algorithms that assume a numerical sequence .

A confusion matrix provides insights into a model's performance by detailing true positives, true negatives, false positives, and false negatives. This breakdown allows for a comprehensive analysis of where the model succeeds or fails, particularly highlighting areas like false positives and negatives, thus facilitating targeted improvements in model tuning .

The random forest classifier is preferred because it is an ensemble method that builds multiple decision trees and averages their predictions, which increases model robustness and prevents overfitting. In contrast, a single decision tree is prone to capturing noise specific to the training data, leading to overfitting. The averaging effect of random forests helps in achieving a more generalized model .

You might also like