0% found this document useful (0 votes)
2 views32 pages

Module 3 Feature Engineering

Module 3 focuses on Feature Engineering, which involves creating, selecting, transforming, and improving raw data into meaningful features for Machine Learning models. It covers various types of feature engineering, including feature creation, transformation, selection, and extraction, along with their importance in enhancing model performance. The module also discusses feature importance methods and transformations like log, square root, and reciprocal transformations to handle data issues and improve model accuracy.

Uploaded by

saararaihana
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
2 views32 pages

Module 3 Feature Engineering

Module 3 focuses on Feature Engineering, which involves creating, selecting, transforming, and improving raw data into meaningful features for Machine Learning models. It covers various types of feature engineering, including feature creation, transformation, selection, and extraction, along with their importance in enhancing model performance. The module also discusses feature importance methods and transformations like log, square root, and reciprocal transformations to handle data issues and improve model accuracy.

Uploaded by

saararaihana
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

MODULE 3 – FEATURE ENGINEERING

1. Feature Engineering Overview


Feature Engineering is the process of creating, selecting, transforming, and improving raw data into
meaningful features that can be effectively used by Machine Learning models. It converts raw data
into a structured format that helps algorithms understand patterns easily. The goal is to improve
model performance by providing better input data. Good feature engineering directly increases
accuracy and reduces errors.
Types of Feature Engineering

1. Feature Creation: Creating new features from existing data.

• Example: Income − Expense = Savings

2. Feature Transformation: Changing data into a better format.

• Example: Log, Scaling, Encoding

3. Feature Selection: Selecting only important features.

• Example: Keep Age, Salary → Remove ID, Name

4. Feature Extraction: Combining multiple features into new ones.

• Example: PCA, BMI calculation

Example

Raw Data:
Date Salary City
2026-06-19 50000 Chennai

Engineered Features:
• Day = Friday
• Month = June
• Salary scaled

Feature Creation
package module3.pkg01;
import [Link];
import [Link];
public class CSVReaderDemo {
public static void main(String[] args) {
String file =
"[Link]";
1
try {
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
// Skip header row
[Link]();
[Link]("FEATURE ENGINEERING OUTPUT");
[Link]("------------------------------------------");
while((line = [Link]()) != null)
{
String[] data = [Link](",");
// Check column count
if([Link] < 3)
{
continue;
}
// Skip empty Data_value
if(data[2].trim().isEmpty())
{
continue;
}
String period = data[1];
double value = [Link](data[2]);
// Feature Transformation
double valueInThousands = value / 1000;
[Link]("Period: " + period +" | Original: " + value + " | Thousands: " +
valueInThousands);
}
[Link]();
[Link]("\nFeature Engineering Completed Successfully");
}catch(Exception e){
[Link]();
}
}}

Output

2
2. Feature Selection – Useful vs Irrelevant Features

Feature Selection is the process of selecting only the most important features from a dataset while
removing irrelevant or unnecessary features. Useful features help improve prediction accuracy, while
irrelevant features add noise and reduce model performance. Feature Selection helps reduce
overfitting, improves model efficiency, reduces training time, and increases accuracy.

Types of Feature Selection

1. Filter Method

The Filter Method uses statistical techniques to select important features before training the model.

Examples:

• Correlation
• Chi-Square Test
• Information Gain

2. Wrapper Method

The Wrapper Method uses Machine Learning algorithms to test different combinations of features
and select the best set of features.

Examples:

• Forward Selection
• Backward Elimination
• Recursive Feature Elimination (RFE)

3. Embedded Method

In the Embedded Method, feature selection is performed automatically during model training.

Examples:

• Decision Tree
• Random Forest
• Lasso Regression

Dataset

Age Salary ID Name Result


25 50000 101 A Yes

Feature Selection
3
Useful Features:

Age , Salary

Irrelevant Features:

• ID, Name

Example

package module3.pkg01;
import [Link];
import [Link];
public class FeatureSelectionDemo {
public static void main(String[] args) {
String file = "[Link]";
try {
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
// Skip Header
[Link]();
[Link]("FEATURE SELECTION OUTPUT");
[Link]("--------------------------------");
while ((line = [Link]()) != null) {
String[] data = [Link](",");
// Check minimum columns
if ([Link] < 3) {
continue;
}
// Skip empty values
if (data[2].trim().isEmpty()) {
continue;
}
// Useful Features
String period = data[1];
String dataValue = data[2];
[Link]("Useful Features");
[Link]("Period : " + period);
[Link]("Data Value : " + dataValue);
// Irrelevant Features
[Link]("Ignored Features");
[Link]("Series Reference : " + data[0]);
[Link]("--------------------------------");
}
[Link]();
[Link]("Feature Selection Completed Successfully");
} catch (Exception e) {
4
[Link]();
}
}
}

Output

3. Importance of Features in ML
Feature importance refers to measuring how much each feature contributes to the prediction made
by a machine learning model. Some features strongly affect output, while others have little or no
impact. Identifying important features helps improve model accuracy and interpretability. It also
helps remove unnecessary features and reduce complexity.
Types of Feature Importance Methods

1. Model-Based Importance

Uses ML models like Decision Tree, Random Forest.

2. Permutation Importance

Measures performance drop when feature is shuffled.

3. Statistical Importance

Uses correlation or statistical scores.

Example
Feature Importance
Area High
Location High
5
Age Medium
ID None

[Link]
ID,StudyHours,Attendance,Marks

1,2,60,45

2,4,70,55

3,5,80,65

4,6,85,72

5,8,95,90

6,7,90,85

7,3,65,50

8,9,98,95

Example code
package module3.pkg01;
import [Link];
import [Link];
import [Link];
public class FeatureImportanceDemo {
// Method to calculate Pearson Correlation
public static double correlation(ArrayList<Double> x, ArrayList<Double> y) {
int n = [Link]();
double sumX = 0;
double sumY = 0;
double sumXY = 0;
double sumX2 = 0;
double sumY2 = 0;

for(int i=0;i<n;i++){
sumX += [Link](i);
sumY += [Link](i);
sumXY += [Link](i) * [Link](i);
sumX2 += [Link](i) * [Link](i);
sumY2 += [Link](i) * [Link](i);
}
6
double numerator = (n * sumXY) - (sumX * sumY);
double denominator = [Link](((n * sumX2) - (sumX * sumX)) * ((n * sumY2) - (sumY *
sumY)));
return numerator / denominator;
}
public static void main(String[] args) {
String file = "[Link]";
ArrayList<Double> studyHours = new ArrayList<>();
ArrayList<Double> attendance = new ArrayList<>();
ArrayList<Double> marks = new ArrayList<>();
try{
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
[Link](); // Skip Header
while((line = [Link]()) != null){
String[] data = [Link](",");
[Link]([Link](data[1]));
[Link]([Link](data[2]));
[Link]([Link](data[3]));
}
[Link]();
double studyImportance = [Link](correlation(studyHours, marks));
double attendanceImportance = [Link](correlation(attendance, marks));
[Link]("===== FEATURE IMPORTANCE =====");
[Link]();
[Link]("Study Hours Importance : %.3f\n", studyImportance);
[Link]("Attendance Importance : %.3f\n", attendanceImportance);
[Link]();
if(studyImportance > attendanceImportance){
[Link]("Most Important Feature : Study Hours");
}
else{
[Link]("Most Important Feature : Attendance");
}
}
catch(Exception e){
[Link](e);
}
}
}
7
Output

FEATURE TRANSFORMATION

1. Feature Transformation
Feature Transformation is the process of modifying, converting, or mathematically transforming
existing features into a new form that helps Machine Learning algorithms understand the data better.
The transformed feature contains the same information as the original feature but in a format that is
easier for the model to process.

Why Do We Need Feature Transformation?

Real-world datasets often have several problems:

1. Skewed Data
Some values occur much more frequently than others.
Example:

Income
20000
25000
30000
35000
500000

The last value is much larger than the others, causing a skewed distribution.

8
2. Large Numerical Value
Large numbers may dominate calculations and make training difficult.

Example:
House Price
1000000
2500000
5000000

3. Presence of Outliers
Outliers are unusually large or small values.
Example:
Age
20
25
30
35
150

The value 150 is an outlier.

4. Non-Normal Distribution
Many Machine Learning algorithms perform better when data follows a normal distribution.
Feature transformation helps make data closer to a normal distribution.

Goals of Feature Transformation

• Reduce skewness
• Reduce variance
• Handle outliers
• Improve model accuracy
• Improve training speed
• Make data more normally distributed
• Improve feature relationship

Types of Feature Transformation

1. Log Transformation
Log Transformation is a mathematical technique used to reduce the impact of very large values in
a dataset. It converts data into a smaller scale using logarithmic functions, making the distribution
9
more balanced. This transformation is commonly applied to highly skewed data where a few values
are much larger than the rest. It helps machine learning algorithms learn patterns more effectively.

Formula: 𝑿′ = 𝒍𝒐𝒈(𝑿)
Example

Original Value Log Value


10 1
100 2
1000 3
10000 4

Notice that very large values become much smaller

Company annual sales:

Sales
1000
5000
10000
50000

The value 500000 is extremely large.

After Log Transformation:


Sales:
3
3.7
4
5.7
The values become more balanced

Advantages

• Reduces skewness
• Compresses large values
• Improves model performance
• Handles outliers better
Log Transformation –Example Code

package module3.pkg01;
import [Link];
10
import [Link];
public class LogTransformation {
public static void main(String[] args) {
String file = "salary_data.csv";
try {
BufferedReader br = new BufferedReader(new FileReader(file));
[Link](); // skip header
String line;
[Link]("LOG TRANSFORMATION");

while ((line = [Link]()) != null) {


String[] data = [Link](",");
double salary = [Link](data[1]);
double logValue = Math.log10(salary);
[Link]("Salary: " + salary + " -> Log: " + logValue);
}
[Link]();
} catch (Exception e) {
[Link]();
}
}
}

OUTPUT

Square Root Transformation

Square Root Transformation is used to reduce moderate skewness and variance in numerical data. It
transforms each value into its square root, which helps compress larger values while preserving the
overall structure of the data. This transformation is less aggressive than Log Transformation and is
useful when data contains moderate outliers. It can improve the distribution of data and model
performance

Formula: 𝑿′ = √𝑿

11
Example
Original Value Square Root
25 5
64 8
100 10

Advantages

• Reduces variance
• Stabilizes data distribution
• Less aggressive than Log Transformation

Example Code
package module3.pkg01;
import [Link];
import [Link];
public class SqrtTransformation {
public static void main(String[] args) {
String file = "salary_data.csv";
try {
BufferedReader br = new BufferedReader(new FileReader(file));
[Link]();
String line;
[Link]("SQUARE ROOT TRANSFORMATION");
while ((line = [Link]()) != null) {
String[] data = [Link](",");
double salary = [Link](data[1]);
double result = [Link](salary);
[Link]("Salary: " + salary +
" -> Sqrt: " + result);
}
[Link]();
} catch (Exception e) {
[Link]();
}
}
}

Output

12
1. Reciprocal Transformation

Reciprocal Transformation converts each value into its inverse by taking 1 divided by the original
value. It is mainly used when datasets contain extremely large values or strong positive skewness.
This transformation significantly reduces the influence of outliers and helps create a more balanced
data distribution. It is useful in situations where large values dominate the dataset.
𝟏
Formula: 𝑿′ =
𝑿

Example

Original Value Reciprocal


10 0.1
20 0.05
100 0.01

Advantages

• Reduces influence of extreme values


• Useful for highly skewed data

Comparison of Transformations

Transformation Purpose
Log Reduce high skewness
Square Root Reduce moderate skewness
Reciprocal Handle extreme values
Scaling Bring values to same range

Example Code
package module3.pkg01;

import [Link];
import [Link];

public class ReciprocalTransformation {


public static void main(String[] args) {
String file = "salary_data.csv";
try {
BufferedReader br = new BufferedReader(new FileReader(file));
[Link]();
String line;
[Link]("RECIPROCAL TRANSFORMATION");
while ((line = [Link]()) != null) {
String[] data = [Link](",");
13
double salary = [Link](data[1]);
double result = 1 / salary;
[Link]("Salary: " + salary + " -> Reciprocal: " + result);
}
[Link]();
} catch (Exception e) {
[Link]();
}
}
}

Output

2. Feature Scaling

Feature Scaling is the process of adjusting the range of numerical features so that all variables
contribute equally during model training. Different features often have different scales, and larger
values can dominate smaller values in some algorithms. Scaling ensures fair comparison among
features and improves the efficiency of learning algorithms. It is especially important for distance-
based machine learning models.

Why Feature Scaling is Important


Consider the following dataset:

Age Salary
25 50000
30 60000
35 70000

Age ranges from:

0 to 100

14
Salary ranges from:

0 to 100000

The salary values are much larger.

Some algorithms calculate distance between data points. Larger values can dominate smaller
values and lead to incorrect predictions.

Benefits of Feature Scaling

• Faster training
• Better accuracy
• Equal importance to features
• Improved convergence
• Better optimization

Types of Scaling

1. Min-Max Scaling

Feature Scaling is the process of adjusting the range of numerical features so that all variables
contribute equally during model training. Different features often have different scales, and larger
values can dominate smaller values in some algorithms. Scaling ensures fair comparison among
features and improves the efficiency of learning algorithms. It is especially important for distance-
based machine learning models.

𝑿−𝑴𝒊𝒏
Formula: 𝑿′ =
𝑴𝒂𝒙−𝑴𝒊𝒏

Example

Original values:

10, 20, 30

After scaling:

0, 0.5, 1

Advantages

• Easy to understand
• Preserves relationships
• Values remain in fixed range
15
Example Code
package module3.pkg01;

import [Link];
import [Link];
import [Link];
import [Link];

public class MinMaxScaling {


public static void main(String[] args) {
String file = "salary_data.csv";
ArrayList<Double> exp = new ArrayList<>();
try {
BufferedReader br = new BufferedReader(new FileReader(file));
[Link]();
String line;
while ((line = [Link]()) != null) {
String[] data = [Link](",");
[Link]([Link](data[0]));
}
double min = [Link](exp);
double max = [Link](exp);
[Link]("MIN-MAX SCALING (Experience)");
for (double v : exp) {
double scaled = (v - min) / (max - min);
[Link]("Experience: " + v +" -> Scaled: " + scaled);
}
[Link]();
} catch (Exception e) {
[Link]();
}
}
}

Output

16
2. Standard Scaling

Standard Scaling is a technique that transforms data so that it has a mean of 0 and a standard
deviation of 1. It measures how far each value is from the average value of the dataset. This method
is useful when features have different units or scales. Many machine learning algorithms perform
better when data is standardized using this approach.

𝑿−𝝁
Formula: 𝒁 =
𝝈

Where:

• μ = Mean
• σ = Standard Deviation
Result

After scaling:
Mean = 0
Standard Deviation = 1

Advantages

• Handles wide value ranges


• Suitable for many ML algorithms
• Works well with normally distributed data
Algorithms That Require Scaling

These algorithms depend on distance calculations.


Algorithm
KNN
K-Means SVM
Logistic Regression
Neural Networks PCA

Algorithms That Do Not Require Scaling

These algorithms split data using rules rather than distances.


Algorithm
Decision Tree
Random
Forest
XGBoost
LightGBM

Example Code
17
package module3.pkg01;

import [Link];
import [Link];
import [Link];

public class StandardScaling {

public static void main(String[] args) {


String file = "salary_data.csv";
ArrayList<Double> sal = new ArrayList<>();
try {
BufferedReader br = new BufferedReader(new FileReader(file));
[Link]();
String line;
while ((line = [Link]()) != null) {
String[] data = [Link](",");
[Link]([Link](data[1]));
}
double sum = 0;
for (double v : sal) sum += v;
double mean = sum / [Link]();
double var = 0;
for (double v : sal)
var += [Link](v - mean, 2);
double std = [Link](var / [Link]());
[Link]("STANDARD SCALING (Salary)");
for (double v : sal) {
double z = (v - mean) / std;
[Link]("Salary: " + v + " -> Z-score: " + z);
}
} catch (Exception e) {
[Link]();
}
}

Output

18
1. Label Encoding
Label Encoding is a technique used to convert categorical text values into numerical values. Since
machine learning algorithms work with numbers rather than text, categories must be represented
numerically before training. Each unique category is assigned a unique integer value. It is simple,
memory-efficient, and commonly used for ordinal categorical data.

Example

Before Encoding

Color
Red
Blue
Green
After Encoding

Color Value
Red 0
Blue 1
Green 2

Why Label Encoding?

Computers work with numbers, not words.

Therefore, text categories must be transformed into numerical form before model training.

Advantages

• Simple implementation
• Low memory usage
• Faster processing
• Useful for ordinal data

Disadvantages

The model may assume an order exists.

Example:
Education University = 2
School = 0
College = 1
19
This order makes sense.
But for:
Gender
Male = 0
Female =1
There is no actual ranking.

Example Code
package module3.pkg01;
import [Link];
import [Link];

public class LabelEncoding {


public static void main(String[] args) {
String file = "salary_data.csv";
try {
BufferedReader br = new BufferedReader(new FileReader(file));
[Link]();
String line;
[Link]("LABEL ENCODING (Experience Level)");
while ((line = [Link]()) != null) {
String[] data = [Link](",");
double exp = [Link](data[0]);
String level = exp < 2 ? "Junior" : exp < 3 ? "Mid" : "Senior";
int encoded = [Link]("Junior") ? 0 : [Link]("Mid") ? 1 : 2;
[Link](level + " -> " + encoded);
}
[Link]();
} catch (Exception e) {
[Link]();
}
}
}

Output

20
One-Hot Encoding

One-Hot Encoding is a method of converting categorical data into multiple binary columns.
Instead of assigning a single number to each category, it creates a separate column for every unique
category. The presence of a category is represented by 1, while its absence is represented by 0.
This approach avoids creating unwanted relationships between categories and is widely used in
machine learning.

Gender
Male
Female
Male

After Encoding
Male Female
1 0
0 1
1 0
Advantages
• No false ordering
• Better for nominal categories
• Widely used in Machine Learning
Example Code
package module3.pkg01;
import [Link];
import [Link];
21
public class OneHotEncoding {
public static void main(String[] args) {
String file = "salary_data.csv";
try {
BufferedReader br = new BufferedReader(new FileReader(file));
[Link]();
String line;
[Link]("ONE HOT ENCODING (Level)");
while ((line = [Link]()) != null) {
String[] data = [Link](",");
double exp = [Link](data[0]);
String level = exp < 2 ? "Junior" : exp < 3 ? "Mid" : "Senior";
int j = [Link]("Junior") ? 1 : 0;
int m = [Link]("Mid") ? 1 : 0;
int s = [Link]("Senior") ? 1 : 0;
[Link](level + " -> " + j + " " + m + " " + s);
}
[Link]();
} catch (Exception e) {
[Link]();
}
}
}

22
Output

23
Feature Engineering
1. Feature Normalization
Definition
Feature Normalization is the process of scaling data values into a fixed range (usually 0 to 1) so
that all features contribute equally to the model.
Why Normalization is Needed?
In datasets, features may have different ranges:
• Salary → 50,000
• Age → 20
• Experience → 5
Large values (Salary) dominate small values (Age) Normalization fixes this problem.
𝑋−𝑋𝑚𝑖𝑛
Formula (Min-Max Scaling): 𝑋 ′ = 𝑋
𝑚𝑎𝑥 −𝑋𝑚𝑖𝑛

Example

Value Normalized
10 0.0
20 0.5
30 1.0
Advantages
• Brings all features to same scale
• Improves model performance
• Useful for KNN, Neural Networks

Disadvantages
• Sensitive to outliers
• Changes original data distribution

Example code:

package module3.pkg01;
import [Link].*;

public class Normalization {


public static void main(String[] args) {
24
try {
BufferedReader br = new BufferedReader(new FileReader("[Link]"));
String line = [Link](); // header
double min = Double.MAX_VALUE;
double max = Double.MIN_VALUE;
String[] store = new String[200];
int i = 0;
while ((line = [Link]()) != null) {
String[] data = [Link](",");
double val = [Link](data[0]);
store[i++] = [Link](val);
if (val < min) min = val;
if (val > max) max = val;
}
[Link]("MIN-MAX NORMALIZATION");
for (int j = 0; j < i; j++) {
double v = [Link](store[j]);
double norm = (v - min) / (max - min);
[Link](v + " -> " + norm);
}
} catch (Exception e) {
[Link]();
}
}
}

Output

25
What it does:
• Converts all values into range 0 to 1
• Example: Salary, Age, Experience scaled equally

2. Standardization
Standardization is a data preprocessing technique used in Machine Learning to transform
numerical values so that the dataset has a mean of 0 and a standard deviation of 1. It converts
each value into a Z-score, which indicates how far the value is from the average of the
dataset. Standardization helps features with different scales contribute equally to the model
and improves the performance of algorithms such as Linear Regression, Logistic Regression,
SVM, and Neural Networks.
Mean = 0
Standard Deviation = 1
𝒙−𝝁
Formula (Z-Score): 𝒛 = 𝝈

Explanation
Each value shows how far it is from the mean.
• Positive value → above mean
• Negative value → below mean
Example
If:
• Mean = 50
• Standard Deviation = 10
Then:
• 60 → (60-50)/10 = 1
• 40 → (40-50)/10 = -1
Advantages
• Works well with Gaussian data
• Less affected by outliers than normalization
• Preferred in Linear Regression, Logistic Regression, SVM
Disadvantages
• Does not bound values (can be negative or >1)

26
• Harder to interpret

Difference: Normalization vs Standardization

Feature Normalization Standardization


Range 0 to 1 No fixed range
Method Min-Max Z-score
Best for Neutral networks, KNN Linear models, SVM

Example Code

package module3.pkg01;
import [Link].*;
import [Link].*;

public class Standardization {


public static void main(String[] args) {
try {
BufferedReader br = new BufferedReader(new FileReader("[Link]"));
[Link]();
ArrayList<Double> list = new ArrayList<>();
String line;
while ((line = [Link]()) != null) {
String[] data = [Link](",");
[Link]([Link](data[0]));
}
double sum = 0;
for (double v : list) sum += v;
double mean = sum / [Link]();
double var = 0;
for (double v : list)
var += [Link](v - mean, 2);
double std = [Link](var / [Link]());
[Link]("STANDARDIZATION (Z-SCORE)");
for (double v : list) {
double z = (v - mean) / std;
[Link](v + " -> " + z);
}
} catch (Exception e) {
[Link]();
}
}
}

27
Output

What it does:
Converts data into:
Mean = 0
Standard deviation = 1

3. Dimensionality Reduction Basics


Dimensionality Reduction is a data preprocessing technique in Machine Learning in which the
number of input variables (features or columns) in a dataset is reduced by transforming or selecting
only the most important information, while preserving the essential patterns, relationships, and
structure of the original data.

Why it is needed?
Real datasets may have:
• 100 features
• 1000 features
• Even more
Problems:
• Slow training

• Overfitting
• High complexity
28
Example
Before:
• Age
• Salary
• Experience
• Education level
• City
• Gender
After reduction:
• 2 or 3 important combined features

Advantages
• Faster training
• Less memory usage
• Reduces overfitting
• Removes noise
Disadvantages
• Some information is lost
• Hard to interpret new features

Example code

package module3.pkg01;
import [Link];
import [Link];

public class DimensionalityReduction {


public static void main(String[] args) {
String file = "[Link]";
try {
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
// Skip Header
[Link]();
[Link]("DIMENSIONALITY REDUCTION");
[Link]("----------------------------------");

29
[Link]("Original Features:");
[Link]("SepalLength, SepalWidth, PetalLength, PetalWidth");
[Link]();
[Link]("Reduced Features:");
[Link]("SepalLength, PetalLength");
[Link]("----------------------------------");

while ((line = [Link]()) != null) {


String[] data = [Link](",");
double sepalLength = [Link](data[0]);
double petalLength = [Link](data[2]);
[Link]("SepalLength: " + sepalLength + " | PetalLength: " +
petalLength);
}
[Link]();
} catch (Exception e) {
[Link]();
}
}
}
Output

What it does:
• Reduces number of columns
• Keeps only important features

4. PCA Overview (Principal Component Analysis)


Principal Component Analysis (PCA) is a statistical and machine learning technique used for
30
dimensionality reduction that transforms a large set of possibly correlated input features into
a smaller set of new variables called Principal Components, which are uncorrelated with each
other and are arranged in such a way that the first few components capture the maximum
amount of variance (information) present in the original dataset.
How PCA Works (Intuition)
1. Finds patterns in data
2. Detects correlation between features
3. Creates new axes (principal components)
4. First component = most important information
5. Second = next important, and so on
Example
Original features:
• Height
• Weight
• BMI (correlated)
PCA converts into:
• PC1 (body size factor)

• PC2 (variation factor)


Key Idea
• PC1 → maximum information
• PC2 → second maximum
• Others → less important

Advantages of PCA
• Reduces high-dimensional data
• Removes multicollinearity
• Improves model speed
• Useful for visualization (2D/3D)

Disadvantages
• Hard to interpret new features
• Sensitive to scaling
• Information loss possible
Example Code
package module3.pkg01;
import [Link];
import [Link];
import [Link];
public class PCAOverview {
public static void main(String[] args) {
String file = "[Link]";
ArrayList<Double> pc1List = new ArrayList<>();

31
ArrayList<Double> pc2List = new ArrayList<>();
try {
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
[Link](); // Skip Header
while ((line = [Link]()) != null) {
String[] data = [Link](",");
double sepalLength = [Link](data[0]);
double sepalWidth = [Link](data[1]);
double petalLength = [Link](data[2]);
double petalWidth = [Link](data[3]);
// Simulated Principal Components
double pc1 = (sepalLength + petalLength) / 2.0;
double pc2 = (sepalWidth + petalWidth) / 2.0;
[Link](pc1);
[Link](pc2);
}
[Link]();
[Link]("PCA OVERVIEW");
[Link]("PC1\tPC2");
for (int i = 0; i < [Link](); i++) {
[Link]( [Link](i) + "\t" + [Link](i));
}
} catch (Exception e) {
[Link]();
}
}
Output

What it does:
• Converts correlated features into: PC1, PC2, PC3...
• Keeps maximum variance

32

You might also like