Normalization

Lecture # 05
Normalization
Normalization is a crucial step in data preprocessing, especially in the context of machine
learning and data analysis. It involves scaling data so that it fits within a specific range or
distribution, making it easier to work with and often improving the performance of
algorithms. Here are some common techniques for normalization:
► Min-Max Normalization (Rescaling):
► Z-Score Normalization (Standardization):
► Robust Scaling:
► Log Transformation:
The choice of normalization technique depends on the nature of the data and the
requirements of the machine learning algorithms you’re using. For instance, algorithms like
k-means clustering and gradient descent are sensitive to the scale of the data, so
normalization can significantly impact their performance.
Common techniques for normalization:
Normalization is a crucial step in data preprocessing, especially in the context of machine learning and data
analysis. It involves scaling data so that it fits within a specific range or distribution, making it easier to work
with and often improving the performance of algorithms. Here are some common techniques for normalization:
► Min-Max Normalization (Rescaling):
Formula: x' = {x - x_min}/{x_max - x_min}
Scales the data to a fixed range, typically [0, 1]. It’s sensitive to outliers because the range is determined by the
minimum and maximum values.
► Z-Score Normalization (Standardization):
Formula: x' = {x - u}/{sigma} (here u is standard deviation and sigma is variance)
Transforms data to have a mean of 0 and a standard deviation of 1. It’s less sensitive to outliers than min-max
normalization but still considers the distribution of the data.

► Robust Scaling:
Formula: x' = {x - Median}/{IQR}
Uses the median and the interquartile range (IQR) instead of the mean and standard deviation. This technique is
more robust to outliers.

► Log Transformation:
Formula: x' = log(x + 1)
Useful for reducing the effect of skewness in the data, especially for data with exponential growth or large
ranges. The "+1" is used to handle zero values.

The choice of normalization technique depends on the nature of the data and the requirements of the machine
learning algorithms you’re using. For instance, algorithms like k-means clustering and gradient descent are
sensitive to the scale of the data, so normalization can significantly impact their performance.
 Min-Max Normalization (Rescaling):

Conversion Avg. Order Value Return Rate

Entry Sales (in $) Visits
Rate (in $) (%)
1 5000 200 0.05 100 10
2 7000 250 0.07 120 12
3 6000 180 0.06 110 8
4 8000 220 0.08 130 9
5 6500 210 0.06 115 11
Min-Max Normalization Steps

► Find the minimum and maximum values for each feature:

► Apply the min-max normalization formula to each value:
[Link] the minimum and maximum values for
each feature:

► Sales (in $): Min = 5000, Max = 8000

► Visits: Min = 180, Max = 250
► Conversion Rate: Min = 0.05, Max = 0.08
► Avg. Order Value (in $): Min = 100, Max = 130
► Return Rate (%): Min = 8, Max = 12
[Link] the min-max normalization formula to
each value:

► Formula:
x' = {x - x_min}/{x_max - x_min}
Sales (in $):

Entry 1: {5000 - 5000}/{8000 - 5000} = {0}{3000} = 0

Entry 2: {7000 - 5000}/{8000 - 5000} = {2000}/{3000} ~ 0.67
Entry 3: {6000 - 5000}/{8000 - 5000} = {1000}/{3000} ~ 0.33
Entry 4: {8000 - 5000}/{8000 - 5000} = {3000}/{3000} = 1
Entry 5: {6500 - 5000}/{8000 - 5000} = {1500}/{3000} = 0.5
Visits:

Entry 1: {200 - 180}/{250 - 180} = {20}/{70} ~ 0.29

Entry 2: {250 - 180}/{250 - 180} = {70}/{70} = 1
Entry 3: {180 - 180}/{250 - 180} = {0}/{70} = 0
Entry 4: {220 - 180}/{250 - 180} = {40}/{70} ~ 0.57
Entry 5: {210 - 180}/{250 - 180} = {30}/{70} ~ 0.43
Conversion Rate:
Entry 1: {0.05 - 0.05}/{0.08 - 0.05} = {0}/{0.03} = 0
Entry 2: {0.07 - 0.05}/{0.08 - 0.05} = {0.02}/{0.03} ~ 0.67
Entry 3: {0.06 - 0.05}/{0.08 - 0.05} = {0.01}/{0.03} ~ 0.33
Entry 4: {0.08 - 0.05}/{0.08 - 0.05} = {0.03}/{0.03} = 1
Entry 5: {0.06 - 0.05}/{0.08 - 0.05} = {0.01}/{0.03} ~ 0.33
Avg. Order Value (in $):
Entry 1: {100 - 100}/{130 - 100} = {0}/{30} = 0
Entry 2: {120 - 100}/{130 - 100} = {20}/{30} ~ 0.67
Entry 3: {110 - 100}/{130 - 100} = {10}/{30} ~ 0.33
Entry 4: {130 - 100}/{130 - 100} = {30}/{30} = 1
Entry 5: {115 - 100}/{130 - 100} = {15}/{30} = 0.5
Return Rate (%):
Entry 1: {10 - 8}/{12 - 8} = {2}/{4} = 0.5
Entry 2: {12 - 8}/{12 - 8} = {4}/{4} = 1
Entry 3: {8 - 8}/{12 - 8} = {0}/{4} = 0
Entry 4: {9 - 8}/{12 - 8} = {1}/{4} = 0.25
Entry 5: {11 - 8}/{12 - 8} = {3}/{4} = 0.75
Normalized Matrix
The normalized dataset is:

Conversion Avg. Order Value Return Rate

Entry Sales (in $) Visits
Rate (in $) (%)
1 0 0.29 0 0 0.5
2 0.67 1 0.67 0.67 1
3 0.33 0 0.33 0.33 0
4 1 0.57 1 1 0.25
5 0.5 0.43 0.5 0.5 0.75
Sample task

Delivery Customer Quantity Discount

Entry Order Value
Time Rating Ordered Percentage
1 25 30 4.5 3 5
2 40 45 3.8 2 10
3 15 20 4.0 1 0
4 30 35 4.2 4 7
5 50 50 4.7 5 12
Z-Score Normalization (Standardization):
Z-score normalization, or standardization, is a technique used to transform data into a
format where the mean is 0 and the standard deviation is 1. This is particularly useful in
many statistical analyses and machine learning algorithms where assumptions about the
distribution of the data are made.
The Z-score of a data point is calculated as:
z = {x - µ}/{σ}
where:
• x is the original value.
• µ (mu) is the mean of the data.
• σ (sigma) is the standard deviation of the data.
Steps for Z-Score Normalization

► Calculate the Mean and Standard Deviation for Each Feature

► Compute the Z-Score for Each Value

z = {x - µ}/{σ}
[Link] the Mean and Standard Deviation
for Each Feature
►
[Link] the Z-Score for Each Value
Use the formula z = {x - µ}/{σ} to normalize each value.
Order Value:
• Entry 1: z = {25 - 32}/{13.53} ~ -0.52
• Entry 2: z = {40 - 32}/{13.53} ~ 0.59
• Entry 3: z = {15 - 32}/{13.53} ~ -1.26
• Entry 4: z = {30 - 32}/{13.53} ~ -0.15
• Entry 5: z = {50 - 32}/{13.53} ~ 1.33
Delivery Time:

Entry 1: z = {30 - 36}/{11.40} ~ -0.53

Entry 2: z = {45 - 36}/{11.40} ~ 0.79
Entry 3: z = {20 - 36}/{11.40} ~ -1.40
Entry 4: z = {35 - 36}/{11.40} ~ -0.09
Entry 5: z = {50 - 36}/{11.40} ~ 1.23
Customer Rating:
Entry 1: z = {4.5 - 4.04}/{0.33} ~ 1.39
Entry 2: z = {3.8 - 4.04}/{0.33} ~ -0.73
Entry 3: z = {4.0 - 4.04}/{0.33} ~ -0.12
Entry 4: z = {4.2 - 4.04}/{0.33} ~ 0.48
Entry 5: z = {4.7 - 4.04}/{0.33} ~ 2.00
Quantity Ordered:
Entry 1: z = {3 - 3}/{1.58} = 0
Entry 2: z = {2 - 3}/{1.58} ~ -0.63
Entry 3: z = {1 - 3}/{1.58} ~ -1.27
Entry 4: z = {4 - 3}/{1.58} ~ 0.63
Entry 5: z = {5 - 3}/{1.58} ~ 1.27
Discount Percentage:
Entry 1: z = {5 - 6.8}/{4.33} ~ -0.42
Entry 2: z = {10 - 6.8}/{4.33} ~ 0.75
Entry 3: z = {0 - 6.8}/{4.33} ~ -1.59
Entry 4: z = {7 - 6.8}/{4.33} ~ 0.05
Entry 5: z = {12 - 6.8}/{4.33} ~ 1.21
Normalized Matrix (Z-Scores)

► This standardized dataset now has each feature with a mean of 0 and a standard
deviation of 1, making it suitable for many types of analysis and machine learning
algorithms.

Delivery Customer Quantity Discount

Entry Order Value
Time Rating Ordered Percentage
1 -0.52 -0.53 1.39 0 -0.42
2 0.59 0.79 -0.73 -0.63 0.75
3 -1.26 -1.40 -0.12 -1.27 -1.59
4 -0.15 -0.09 0.48 0.63 0.05
5 1.33 1.23 2.00 1.27 1.21
Practice problem

Conversion Avg. Order Value Return Rate

Entry Sales (in $) Visits
Rate (in $) (%)
1 5000 200 0.05 100 10
2 7000 250 0.07 120 12
3 6000 180 0.06 110 8
4 8000 220 0.08 130 9
5 6500 210 0.06 115 11
Robust Scaling:
The interquartile range (IQR) is a measure of statistical dispersion that describes the range
within which the middle 50% of a dataset falls. It is particularly useful for identifying
outliers and understanding the spread of data.
The IQR is calculated as follows:
{IQR} = Q_3 - Q_1
where:
• (Q_1) (the first quartile) is the median of the first half of the data (25th percentile).
• (Q_3) (the third quartile) is the median of the second half of the data (75th percentile).
Steps to Calculate IQR
► 1. Order the Data: Sort the dataset in ascending order.
► 2. Find Q1: Determine the median of the first half of the data.
► 3. Find Q3: Determine the median of the second half of the data.
► 4. Calculate IQR: Subtract (Q_1) from (Q_3).
Example
Consider the following dataset:

Data Points
2
3
5
7
8
10
12
15
18
Steps:
► 1. Order the Data: Already ordered.
► 2. Find Q1: The first half is ( {2, 3, 5, 7} ). The median is ( {3 + 5}/{2} = 4 ).
► 3. Find Q3: The second half is ( {10, 12, 15, 18} ). The median is ( {12 + 15}/{2} =
13.5 ).
► 4. Calculate IQR:

{IQR} = Q_3 - Q_1 = 13.5 - 4 = 9.5

Importance of IQR
► Outlier Detection: Values that fall below ( Q_1 - 1.5 * {IQR} ) or above ( Q_3 + 1.5 *
{IQR} ) are considered outliers.
► Robustness: IQR is less affected by extreme values compared to the range, making it a
more reliable measure of spread in skewed distributions.
Practice problem 1
Delivery Customer Quantity Discount
Entry Order Value
Time Rating Ordered Percentage
1 25 30 4.5 3 5
2 40 45 3.8 2 10
3 15 20 4.0 1 0
4 30 35 4.2 4 7
5 50 50 4.7 5 12
Practice problem 2

Conversion Avg. Order Value Return Rate

Entry Sales (in $) Visits
Rate (in $) (%)
1 5000 200 0.05 100 10
2 7000 250 0.07 120 12
3 6000 180 0.06 110 8
4 8000 220 0.08 130 9
5 6500 210 0.06 115 11