Data Preprocessing in Data Mining
Real-world data is often incomplete, noisy, and inconsistent, which can lead to incorrect results
if used directly. Data preprocessing in data mining is the process of cleaning and preparing raw
data so it can be used effectively for analysis and model building.
● Real data contains missing and incorrect values
● Data may come from multiple sources
● Large datasets often have irrelevant information
● Clean data gives better mining results
Steps in Data Preprocessing
Some key steps in data preprocessing are:
Data Cleaning
It is the process of identifying and correcting errors or inconsistencies in the dataset. Data
cleaning, also referred to as data scrubbing or data cleansing, is the process of preparing data for
analysis by identifying and correcting errors, inconsistencies, and inaccuracies. Raw data is
usually full of inaccuracies, outliers, missing entries, duplicates, and noise that can only worsen
the result if not handled correctly. It's essentially like cleaning up a messy room before you can
use it effectively.
Techniques for Cleaning Data in Data Mining
Ignore the Tuples
● Objective: This method entails a rejection of every tuple (row) that contains more
than one or two missing attributes or values.
● Process: Such an approach can only be used where a tuple contains many missing
values and it becomes impossible, or unadvisable, to repair it. This method is very
efficient and advisable when the dataset is large and removal of some of the tuples
does not affect the final result.
● Limitations: It is not very practical again, especially in cases where the dataset is
small or where the missing values form part and parcel of the analysis.
Fill in the Missing Value:
Objective: This technique deals with frequency estimates to impute missing values.
Process: Different approaches can be used to fill in the missing values:
● Manual Input: Manually imputing the missing data by using prior knowledge of the
domain or any other related sources.
● Mean/Median Imputation: Replaces missing values with the average of the
attribute. Replaces missing values with the middle value (useful when outliers exist)
or replaces missing values with the most frequent value.
● Most Likely Value: Imputing missing data using the predictive method or by making
use of some mathematical algorithm to forecast the probable value of the missing
data.
Limitations: While this method is effective, it can be time-consuming and takes the guesswork
into account which may lead to bias.
Binning Method:
Objective: The last process which is involved in data preparation is binning this is used in the
process of managing the noise by placing it in bins or intervals.
Process:
● Sort the data values.
● Split the sorted data into an equal number of bins.
● There are three approaches to performing smoothing – Smoothing by bin means: In
smoothing by bin means, each value in a bin is replaced by the mean value of the bin.
Smoothing by bin median: In this method each bin value is replaced by its bin median
value. Smoothing by bin boundary: In smoothing by bin boundaries, the minimum
and maximum values in a given bin are identified as the bin boundaries. Each bin
value is then replaced by the closest boundary value.
Advantages: This method is a beneficial one when it comes to working with ongoing data as it
simplifies the samples by minimizing noise.
Examples: Sorted data for price (in dollars): 4, 8, 9, 15, 21, 21, 24, 25, 26, 28, 29, 34
Partition using equal frequency approach:
- Bin 1 : 4, 8, 9, 15
- Bin 2 : 21, 21, 24, 25
- Bin 3 : 26, 28, 29, 34
Smoothing by bin means:
- Bin 1: 9, 9, 9, 9
- Bin 2: 23, 23, 23, 23
- Bin 3: 29, 29, 29, 29
Smoothing by bin boundaries:
- Bin 1: 4, 4, 4, 15
- Bin 2: 21, 21, 25, 25
- Bin 3: 26, 26, 26, 34
Smoothing by bin median:
- Bin 1: 9 9, 9, 9
- Bin 2: 23, 23, 23, 23
- Bin 3: 29, 29, 29, 29
Regression:
Objective: Regression techniques are used for imputing missing values and also for flattening
data and this is by estimating the relationship between variables.
Process:
● Linear Regression: A cause variable is used to predict an effect variable.
● Multiple Regression: Another advantage of using multiple independent variables in
making a prediction model is that they give a better estimation of the dependent
variable since many independent variables are used in predicting the dependent
variable.
Advantages: Regression is applied when dealing with gaps within data and disguising random
data is present when there is a high correlation of variables.
Clustering:
Objective: It tends to assemble similar groups of data so that it becomes easier for pattern
recognition or detection of outliers.
Process:
● Categorize data into groups in the given data set depending on similarity.
● Eliminate any observations which cannot be easily classified into any group.
● Applying the clustering method, which makes the disaggregate data simpler
by sorting similar values into one cluster.
Advantages: Physically grouping data can be very useful in that clustering can easily identify
‘outliers’, which if left in the model, could significantly influence the analysis for the worse.
Data Transformation
Data transformation converts data into a suitable form so that data mining algorithms can work
effectively.
● Bring data into a common format
● Improve mining efficiency
● Make data suitable for modeling
Normalization:
Data normalization is a technique used in data mining to transform the values of a dataset into a
common scale. This is important because many machine learning algorithms (KNN, neural
networks, clustering algorithms (like K-Means), and gradient-based models) are sensitive to the
scale of the input features and can produce better results when the data is normalized.
Normalization is used to scale the data of an attribute so that it falls in a smaller range, such as -
1.0 to 1.0 or 0.0 to 1.0. It is generally useful for classification algorithms.
Why do we need Normalization?
Machine learning models often assume that all features contribute equally. Features with
different scales can dominate the model’s behavior if not scaled properly. Using normalization,
we can:
● Ensure Equal Contribution of Features: Prevents features with larger scales from
dominating models that are sensitive to magnitude such as K-Nearest Neighbours or
neural networks.
● Improve Model Performance: Algorithms that rely on distances or similarities (KNN,
K-Means clustering) perform better when features are normalized.
● Accelerate Convergence: Helps gradient-based algorithms like logistic regression or
neural networks converge faster by keeping feature values in a similar range.
● Maintain Interpretability of Scales: By converting all features to a common range,
it’s easier to understand their relative impact on predictions.
Difference Between Normalization and Standardization
Standardization, also called Z-score normalization is a separate technique. It transforms data so
that it has a mean of 0 and a standard deviation of 1.
Feature Normalization (Min-Max) Standardization (Z-score)
Goal Rescale data to a specific Center data to mean 0, SD 1
Feature Normalization (Min-Max) Standardization (Z-score)
range
Range of values Fixed (e.g., 0–1) Not fixed
Effect of outliers Sensitive Less sensitive
Assumes data
No Assumes roughly Gaussian
distribution
Algorithms assuming Gaussian or
Distance-based algorithms
Use case regularization
Example Scaling pixel values to [0,1] Scaling test scores to z-scores
Note: Normalization and Standardization are two distinct feature scaling techniques.
Data Transform Techniques:
● Smoothing
● Aggregation
● Discretization
● Attribute Construction
● Decimal Scaling
● Log Transformation
● One-Hot Encoding
● Label Encoding
● Aggregation
[Link]
Smoothing reduces noise or random variation in data to highlight important patterns or trends. It
is especially useful in time-series and continuous numerical data. Example: Consider you have
noisy data like this -> [5, 7, 6, 20, 7, 8, 6].
Here, the value 20 is an outlier that makes the data look jagged. One simple smoothing method is
replacing each value with the average of its neighbors to reduce sharp jumps. For example,
replacing 20 with the average of its neighbors (6 + 7) / 2 = 6.5 gives smoother data:
[5, 7, 6, 6.5, 7, 8, 6]
Some of the common smoothing methods are:
● Moving averages
● Binning
● Regression-based smoothing
2. Aggregation
Aggregation combines data values to produce summary information. It can merge data from
multiple sources or summarize large datasets. Example: Sales, data may be aggregated to
compute monthly and annual total amounts.
January:1000,February:1200,March:1300
Quarterly Sales = 3500
3. Discretization
It is a process of transforming continuous data into set of small intervals. Most Data
Mining activities in the real world require continuous attributes. Yet many of the existing data
mining frameworks are unable to handle these attributes. Also, even if a data mining task can
manage a continuous attribute, it can significantly improve its efficiency by replacing a constant
quality attribute with its discrete values. Example: Continuous age values: 22, 25, 37, 60.
Discretized into categories:
● 0-25 "Young"
● 26–50 "Middle-aged"
● 50+ "Senior"
So, 22 Young, 37 Middle-aged, 60 Senior
4. Attribute Construction
Where new attributes are created and applied to assist the mining process from the given set of
attributes. This simplifies the original data and makes the mining more efficient.
Example: Original attributes:
● Height (in cm) = 175
● Weight (in kg) = 70
Constructed attribute:
BMI = Weight / (Height in meters)2 = 70 / (1.75)2 = 22.86
5. Normalization
Min-Max Normalization
Min-Max normalization rescales a feature to a specific range, typically [0,1]
X normalized=(X −X min )/(X max −X min )
Where:
● X: The original value to be scaled.
● X min :The minimum value in your dataset.
● X max : The maximum value in your dataset.
For example, if you have an "Age" dataset ranging from 18 to 80, an age of 50 would be scaled
5−18
as: X normalized= = 0.52
80−18
Z-score Normalization
Z-score normalization, also called standardization, transforms data so that it has a mean
(average) of 0 and a standard deviation of 1. This process adjusts data values based on how far
X−μ
they deviate from the mean, measured in units of standard deviation. Z= .
σ
Decimal Scaling
Decimal scaling is a data preprocessing technique that normalizes numeric values by shifting the
decimal point. It scales data so that all values fall within a specific interval, typically between -1
and 1. Example:Assume you have a dataset of house prices: 10, $201, $301, -$401, $501, $601,
$701.
● Max Absolute Value: The largest number is 701.
● Scaling Factor: Because 701 has 3 digits, we set j = 3. Thus, our divisor is 10³ or 1,000.
● Scaled Data: Divide each original number by 1,000.
o Original -401 becomes -0.401
o Original 201 becomes 0.201
o Original 701 becomes 0.701
Log Scaling
Log scaling is a normalization technique that is useful when the data has a skewed distribution.
This technique involves taking the logarithm of the data to reduce the effect of extreme values.
Example: Suppose we have a dataset with a feature "Income" that has a skewed distribution. To
normalize this feature using log scaling, we would take the logarithm of each income value. This
would result in a normalized feature with a more even distribution.
6. One-Hot Encoding:
One-Hot Encoding is a data preprocessing technique used to convert categorical data into a
numerical format that machine learning models can understand. It creates separate binary
columns for each category, where 1 represents the presence of a category and 0 represents its
absence.
● Converts categorical values into binary columns
● Prevents models from assuming an incorrect order between categories
● Improves machine learning model performance
● Helps capture relationships between categorical features
● Required for many machine learning algorithms that accept numerical input only
Example:
Fruit Categorical value of fruit Price
Apple 1 5
Mango 2 10
Apple 1 15
Orange 3 20
The output after applying one-hot encoding on the data is given as follows
Fruit Apple Fruit Mango Fruit Orange Price
1 0 0 5
0 1 0 10
1 0 0 15
0 0 1 20
7. Label Encoding:
Label Encoding is a data preprocessing technique used to convert categorical values into
numerical labels. It assigns a unique integer to each category, allowing machine learning
algorithms to process categorical data in a numerical format. The encoded labels are typically
assigned based on the sorted order of unique categories.
Label Encoding is important because many ML algorithms cannot process string values
(Categorical Data) directly. Hence, converting them into numerical values is essential for model
training. Categorical data is broadly divided into two types:
● Nominal Data: Categories without a natural order, such as colors (Red, Blue, Green).
● Ordinal Data: Categories with a natural order, such as satisfaction levels (Low,
Medium, High).
Label encoding works best for ordinal data, where the assigned numbers reflect the order. But
when you apply it to nominal data, the numbers accidentally create a fake ranking (e.g., Red = 0,
Blue = 1, Green = 2).