Data Encoding
Prepared by
Debasree Mitra
Asst. Professor ,CSE Dept.
SOET, Adamas University
Outline
• Data Encoding
• Why is Data Encoding Important?
• Types of data encoding
• Data Normalization
Data encoding
Data encoding in machine
learning is the process of This is a crucial step in feature
converting raw data, especially engineering that ensures
non-numeric (categorical) data, compatibility with algorithms
into a numerical format that and can significantly impact
machine learning algorithms model performance.
can understand and process.
• Most machine learning models, such as
linear regression and support vector
machines, rely on mathematical
calculations and require all input
variables to be numeric. Encoding helps:
• o Ensure compatibility: It transforms
data into a usable format for
algorithms that cannot handle text or
Why is Data other non-numeric types.
o Improve performance: Properly
Encoding encoded data can lead to improved
predictive accuracy and reduced bias.
Important? o Manage data: Encoding helps manage
high-cardinality features (variables
with many unique categories) and can
be used for data compression to save
memory and bandwidth.
1. Label Encoding
• Label Encoding assigns each category a
unique integer. It is simple and memory-
efficient but may unintentionally imply
an order among categories when none
exists.
• Used in tree-based models like Decision
Trees or XGBoost.
• Pros: Simple and memory-efficient.
• Cons: Introduces implicit order which
may be misinterpreted by non-tree
models when used with nominal data.
2. One-Hot Encoding
• One-Hot Encoding converts categories into
binary columns with each column
representing one category. It prevents false
ordering but can lead to high dimensionality
if there are many unique values.
• Used in linear models, logistic regression and
neural networks.
• Pros: Does not assume order; widely
supported.
• Cons: Can cause high dimensionality and
sparse data when feature has many
categories.
Ordinal Encoding
• Ordinal Encoding maps categories
to integers while preserving their
natural order. This works well for
ordered data like ratings but is not
suitable for nominal variables.
o Used for ordered features like
ratings or education levels.
o Pros: Maintains order; reduces
dimensionality.
o Cons: Not suitable for nominal
categories.
Target Encoding
• Target Encoding also known as
Mean Encoding is a technique where
each category in a feature is replaced
by the mean of the target variable for
that category.
o Useful for high-cardinality
features like ZIP codes or
product IDs.
o Pros: Captures relationship to
target variable.
o Cons: Risk of overfitting, also
must apply smoothing/statistical
techniques.
Binary Encoding
• Binary encoding represents
categories as binary codes and
splits them across multiple
columns. It is efficient for high-
cardinality data but slightly more
complex to implement.
• Applied in high-cardinality
text/NLP tasks to save memory.
• Pros: Reduces dimensionality,
more memory-efficient than one-
hot encoding.
• Cons: Slightly more complex;
requires careful handling of
missing values.
Frequency
Encoding
• Frequency Encoding assigns
categories values based on how
often they occur in the dataset.
It is simple and compact but
can introduce data leakage if
applied improperly.
• Effective in retail, e-commerce
or clickstream data for
popularity trends.
• Pros: Low computational and
storage requirements.
• Cons: Can introduce data
leakage if not handled properly.
Data Normalization
• Data Normalization is the process of scaling numerical data into a standard range so that
different features contribute equally to a machine learning model. It ensures that no single
feature dominates due to differences in magnitude.
Why Normalization is Important?
• Improves model performance
• Speeds up convergence in gradient-based algorithms
• Prevents bias toward larger values
• Essential for distance-based algorithms (KNN, K-Means, SVM)
• Min-Max normalization rescales a feature to a specific range, typically [0, 1]:
The minimum value maps to 0
The maximum value maps to 1
Other values are scaled proportionally
Normalization example:
• Given Data (Marks of Students)
• 10,20,30,40,5010, 20, 30, 40, 5010,20,30,40,50
• Minimum value (Xmin) = 10
Maximum value (Xmax) = 50
Important question
• Why do we need encoding?
• Difference between label encoding and one-hot encoding?
• What happens if we don’t perform encoding?
• What is feature selection?
• How does feature selection reduce overfitting?
• What is the difference between filter and wrapper methods?
• Difference Between Feature Selection and Feature Extraction
• What is normalization?
• What is feature selection?
• Differentiate between label encoding and one-hot encoding.
• What is Min-Max scaling?
• Why is data preprocessing important?