0% found this document useful (0 votes)
16 views43 pages

Feature Engineering for Machine Learning

The document outlines a 4-hour course on feature engineering, covering topics such as the definition of features, the importance of feature selection and extraction, and various techniques like one-hot encoding, normalization, and dealing with missing data. It also discusses visual pattern recognition features, including shape-based descriptors and the Histogram of Oriented Gradients (HOG) algorithm. The course emphasizes the need for informative and discriminative features to improve machine learning model performance.

Uploaded by

hadatalex
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
16 views43 pages

Feature Engineering for Machine Learning

The document outlines a 4-hour course on feature engineering, covering topics such as the definition of features, the importance of feature selection and extraction, and various techniques like one-hot encoding, normalization, and dealing with missing data. It also discusses visual pattern recognition features, including shape-based descriptors and the Histogram of Oriented Gradients (HOG) algorithm. The course emphasizes the need for informative and discriminative features to improve machine learning model performance.

Uploaded by

hadatalex
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Feature engineering

 Duration: 4 hrs
 Outline:
1. Introduction
2. Feature engineering
3. Features in visual pattern recognition
4. Shape-based feature descriptors
Feature engineering

 Duration: 4 hrs
 Outline:
1. Introduction
2. Feature engineering
3. Features in visual pattern recognition
4. Shape-based feature descriptors
Introduction to feature & feature engineering
 Feature:
 an individual measurable property or characteristic of a data
example

 describes the example

 Features are usually numeric.

 Feature engineering: transfer raw data into feature vector

Data  feature vector  ML model


The general framework for Machine Learning
Curse of dimensionality
 Dimensionality: the number of features in feature vector.

 Curse of dimensionality:

 The number of features is very large relative to the number of


observations (examples) in dataset

 Hard to train effective model

 Dimensionality reduction
 Feature selection
 Feature extraction
Feature extraction vs. feature selection

 Feature selection:

 Filtering irrelevant or redundant features from dataset

 Choosing a subset of the original features

 Feature extraction:

 Creating a new smaller set of features

 Getting useful features from existing data

 Feature need to be informative, discriminating and


independent
Feature extraction vs. feature selection
Feature engineering

 Duration: 4 hrs
 Outline:
1. Introduction
2. Feature engineering
3. Features in visual pattern recognition
4. Shape-based feature descriptors
Feature engineering
 One-hot encoding

 Binning

 Normalization

 Standardization

 Dealing with missing feature

 Data imputation techniques


One-hot encoding
 Transform a categorical feature into several binary features

 Example: feature “color” has 3 values “red”, “yellow”, green”

 “red” = 1, “yellow” = 2, “green” = 3

 ?
Binning (bucketing)
 Transform a numerical feature into categorical feature

 Example: feature “age”

 Put all ages between 0 and 5 years-old into one bin

 Put ages from 6 to 10 years-old in the second bin

 Put ages from 11 to 15 years-old in the third bin, and so on.


Normalization

 Converting an actual range of values of a numerical feature


into a standard range of values, typically in the interval [-1, 1] or
[0, 1].

 Example: natural range = [350, 1450]

 Subtracting 350 from every value of the feature

 Dividing the result by 1100  normalized range = [0, 1].


Standardization

 Rescaling the feature values so that they have the properties of


a standard normal distribution with µ = 0 and =1

 Formula:
Standardization or normalization?
 Try two if have time 

 Rule of thumbs:
Dealing with missing features

 Removing the examples with missing features.

 Use data imputation technique


Data imputation techniques

 Technique 1: Replacing the missing value of a feature by an


average value of this feature in the dataset

 Technique 2: Replacing the missing value by the same value


outside the normal range of values.

 Technique 3: Replacing the missing value by a value in the


middle of the range.

…etc…
Feature engineering

 Duration: 4 hrs
 Outline:
1. Introduction
2. Feature engineering
3. Features in visual pattern recognition
4. Shape-based feature descriptors
Image feature extraction
 Purpose:

 To reduce the dimensionality of input image


 To transform each input image into a corresponding multi-
dimension feature vector

 To perform the predefined classification tasks with sufficient


accuracy without using the entire input image

 Requirements:
 Features should extract the most suitable characteristics from
the input image
An example of feature extraction

.
Visual features

 Color-based features
Visual features
 Shape-based features
Visual features
 Texture-based features
Which feature is the best?

 Example: plant recognition

 Plant features: leaf, fruit, flower, root, branch,…

 Leaf features: shape, vein, margin, texture


 No single best feature for a given leaf identity  combination
of different features

 No single best presentation for a given feature  multiple


descriptors to characterize the feature from different
perspectives

 Challenging
Deep learning
 Innovative
Feature engineering

 Duration: 4 hrs
 Outline:
1. Introduction
2. Feature engineering
3. Features in visual pattern recognition
4. Shape-based feature descriptors
Shape-based feature descriptor

 Shape: important

 Good shape descriptor: invariant to geometrical


transformations (rotation, reflection, scaling, translation)

 Types of shape descriptors: simple and morphological shape


descriptor (SMSD), contour-based, region-based
Simple and morphological shape descriptor

 Refer to basic geometric properties of the shape

 Basic descriptor: diameter, major axis length, minor axis length,


area, perimeter, centroid,…

 Morphological descriptor: aspect ratio, perimeter to area ratio,


rectangularity measures, circularity measures,…
Contour-based feature descriptor

 Consider the boundary of a shape and neglect the information


contained in the shape interior

 Ex: CCD (centroid contour distance), Fourier descriptor


computed on CCD.
Contour-based feature descriptor
Region-based feature descriptor

 Take all the pixels within a shape region into account to


obtain the shape representation

 Image moments: statistical descriptor of a shape. Ex: Hu


moments

 Local features: select key points in image. Ex: HOG


(histogram of oriented gradients), SIFT (scale-invariant
feature transform)
Histogram of Oriented Gradients (HOG) ALGORITHM

• HOG stands for histogram of oriented gradients.


• The hog descriptor focuses on structure or shape of
the object.
• It uses magnitude as well as direction of the
gradient to compute the features.
• It generates histogram by using magnitude and
direction of the gradient.
HOG ALGORITHM
HOG ALGORITHM

20

40 70

• Here we calculating gradient magnitude and 70


direction, to calculate pixels intensity we need
• X direction=|40-70|=30
• Y direction=|20-70|=50
• By these values we are calculating magnitude and
direction of the gradient
• By using magnitude and direction we calculate
feature vectors
HOG ALGORITHM
e
HOG ALGORITHM

• Before getting the hog feature and after


concatenating feature vectors we are supposed to
do normalize.
• Suppose we have taken 150*300 pixels and multiply
with 2 to increase the brightness and divided by 2 to
decrease the brightness, then you cant compare
two images without normalization bec’z the pixels
intensity will be changed.
• But if you normalize the feature vectors it is easy to
compare
HOG ALGORITHM
HOG ALGORITHM

• For hog features giving human template and giving output


for convolving with human model
• Then it will predict whether it is human or not.
Image moments
Hu moments feature descriptor
Central moments:

Central normalized moments:

Centroid of the image:

_
x.s(x, y) _
 y.s(x, y)
x x y
,y x y

s(x, y) s(x, y)
x y x y
Hu moments feature (cont)
Id image S1 S2 S3 S4 S5 S6 S7

6
HOG

[Link]

Common questions

Powered by AI

One-hot encoding is used in feature engineering to transform categorical features into a numerical format that machine learning models can process effectively. By converting each categorical value into a separate binary feature indicating the presence (1) or absence (0) of that value, one-hot encoding solves the problem of ordinal encoding, where simply assigning arbitrary numbers to categories could imply an incorrect order or hierarchy among them. This approach ensures that categorical information is accurately and usefully incorporated into the feature set .

The curse of dimensionality refers to the issues that arise when the number of features (dimensions) in a dataset is very large relative to the number of observations. This can make it difficult to effectively train machine learning models as the data becomes sparse, and algorithms may struggle to generalize from the training data to unseen data. Common strategies to address the curse of dimensionality include dimensionality reduction techniques such as feature selection, which filters out irrelevant features, and feature extraction, which creates a new set of features from existing data. These methods help reduce complexity and the potential for overfitting .

The Histogram of Oriented Gradients (HOG) algorithm detects human figures by analyzing the underlying shapes and structures within an image through gradient calculations. It computes the gradient direction and magnitude for each image pixel, creating a histogram that captures the distribution of gradient orientations within localized regions. Normalization of feature vectors is necessary to minimize the effects of variations in illumination and shadowing in the image, allowing for a more consistent and robust comparison between different images. By normalizing the feature vectors, HOG ensures that the detection algorithm is primarily influenced by the layout of gradients rather than extraneous factors .

Choosing appropriate feature descriptors in plant recognition through image analysis involves identifying the specific characteristics that most effectively distinguish between different plant species. This includes selecting descriptors that capture vital aspects such as leaf shape, vein pattern, margin type, and texture. The importance lies in their ability to represent diverse biological features that are crucial for accurately differentiating species. In practice, no single feature is sufficient for plant identification; thus, a combination of multiple descriptors is often necessary to provide a comprehensive representation. This approach facilitates more precise recognition and classification, highlighting the need for robust and contextually relevant feature selection in image-based plant recognition systems .

Feature extraction and feature selection both aim to reduce the dimensionality of data, but they differ fundamentally in their approach. Feature selection involves filtering out irrelevant or redundant features from the dataset and choosing a subset of the original features. In contrast, feature extraction creates a new, smaller set of features by transforming the existing data into a more informative representation. Feature extraction is preferable when the original features need to be combined to provide richer information, while feature selection is more suitable when certain features are clearly irrelevant or repetitive .

Hu moments are a set of seven invariant statistical measures derived from image moments that capture the geometric properties of a shape. These descriptors are invariant to image transformations such as translation, scaling, and rotation, making them powerful tools for feature extraction in image processing. They enable the identification and classification of shapes regardless of their orientation or position within the image. By leveraging these invariant moments, image processing applications can reliably compare and recognize shapes across different conditions and perspectives .

Contour-based shape descriptors focus on the boundary of a shape, ignoring the information contained in its interior. Examples include the centroid contour distance (CCD) and Fourier descriptors. These are useful for analyzing shapes where the outline provides sufficient information, such as simple object detection. Region-based descriptors, like Hu moments and Histogram of Oriented Gradients (HOG), consider all pixels within the shape's region, offering a more holistic representation. They are applied in scenarios requiring detailed texture and structure analysis, such as facial recognition and complex pattern analysis .

In visual pattern recognition, using a combination of different feature types is often necessary because no single feature or representation can adequately capture all the useful information needed for accurate classification. For example, in plant recognition, features like leaf shape, vein pattern, margin, and texture each provide unique information that can contribute to correctly identifying different plant species. By combining various feature descriptors, models can achieve a more comprehensive understanding and improve their classification performance .

Feature engineering involves transforming raw data into a feature vector suitable for use in machine learning models. It is crucial because it converts raw inputs into informative, discriminating, and independent features which aid in model prediction and accuracy. Effective feature engineering can greatly enhance the predictive power of models by allowing them to focus on the most relevant signals while reducing noise and computational complexity. This process becomes particularly important when dealing with large datasets, where the curse of dimensionality might lead to overfitting if features are not carefully chosen and optimized .

Normalization and standardization are feature transformation techniques used to adjust the scale of data. Normalization rescales the values of a feature to a standard range, typically [0, 1] or [-1, 1], making it easier to compare features with different scales. Standardization, on the other hand, transforms the data to have a mean of 0 and a standard deviation of 1, mapping the data to fit a normal distribution. The choice between these methods depends on the specific characteristics of the data and the requirements of the machine learning algorithm being used .

You might also like