Module 3
DESCRIPTIVE MODELING
Introduction to Decriptive Modeling
Summary Of Descriptive Modeling
Predictive Modeling
Descriptive Modeling V/S Predictive Modeling
Data Preparation Issues in Descriptive Modelling
Applying PCA to New Data
➢What Is Principal Component Analysis?
• Principal component analysis (PCA) is a dimensionality reduction and
machine learning method used to simplify a large data set into a
smaller set while still maintaining significant patterns and trends.
• Idea of PCA is simple: Reduce the number of variables of a data
set, while preserving as much information as possible.
FITTING THE PCA ON TRAINING DATA AND DATA VISUALIZATION
Step 1: Load the Iris Data Set
import pandas as pd
url = [Link]
# load dataset into Pandas DataFrame
df = pd.read_csv(url, names=['sepal length','sepal width','petal length','petal
width','target'])
Step 2: Standardize the Data
PCA is affected by scale, so you need to scale the features in your data before applying PCA.
Use StandardScaler to help you standardize the data set’s features onto unit scale (mean =
0 and variance = 1), which is a requirement for the optimal performance of many machine
learning algorithms. If you don’t scale your data, it can have a negative effect on your algorithm.
from [Link]
import StandardScaler features = ['sepal length', 'sepal width', 'petal
length', 'petal width’]
# Separating out the features
x = [Link][:, features].values
# Separating out the target
y = [Link][:,['target']].values
# Standardizing the features
x = StandardScaler().fit_transform(x)
Step 3: PCA Projection to 2D
The original data has four columns (sepal length, sepal width,
petal length and petal width).
The code projects the original data, which is four-dimensional,
into two dimensions. After dimensionality reduction, there
usually isn’t a particular meaning assigned to each principal
component. The new components are just the two main
dimensions of variation
from [Link]
import PCA pca = PCA(n_components=2)
principalComponents = pca.fit_transform(x)
principalDf = [Link](data = principalComponents , columns =
['principal component 1', 'principal component 2'])
finalDf = [Link]([principalDf, df[['target']]], axis = 1)
Concatenating DataFrame along axis = 1.
finalDf is the final DataFrame before plotting the data.
Step 4: Visualize 2D Projection
fig = [Link](figsize = (8,8))
ax = fig.add_subplot(1,1,1)
ax.set_xlabel('Principal Component 1', fontsize = 15) ax.set_ylabel('Principal
Component 2', fontsize = 15)
ax.set_title('2 component PCA', fontsize = 20)
targets = ['Iris-setosa', 'Iris-versicolor', 'Iris-virginica’]
colors = ['r', 'g', 'b’]
for target, color in zip(targets,colors):
indicesToKeep = finalDf['target'] == target
[Link]([Link][indicesToKeep, 'principal component 1'] ,
[Link][indicesToKeep, 'principal component 2'] , c = color , s = 50)
[Link](targets)
[Link]()
Applying PCA to New Data
Summary of Operations & Tools for Applying PCA
PCA FOR DATA INTERPRETATION
•Principal Components: The first principal component (PC1) captures the most
variance, while the second (PC2) captures the second most.
•Clusters: The visualization shows that Iris Setosa forms a well-separated cluster,
while Versicolor and Virginica overlap more.
•Explained Variance: You can check how much variance is retained by each component:
The explained variance tells you how much information (variance) can be attributed to each of the
principal components. This is important because while you can convert four-dimensional space to
a two-dimensional space, you lose some of the variance (information) when you do this. By using
the attribute explained_variance_ratio_, you can see that the first principal component contains
72.77 percent of the variance, and the second principal component contains 23.03 percent of the
variance. Together, the two components contain 95.80 percent of the information.
print(pca.explained_variance_ratio_)
Contribution of Original Features
To understand which original features contribute most to the components:
components = [Link](pca.components_, columns=iris.feature_names,
index=['PC1', 'PC2'])
print(components)
This matrix shows how much each original variable contributes to the PCs. For
example, Petal Length and Petal Width usually have strong influence on PC1.
PCA helps in:
• Reducing the dimensionality for visualization.
• Revealing the structure and patterns in the data.
• Understanding the influence of features.
• In the case of the Iris dataset, PCA clearly shows that Iris Setosa is linearly
separable, and the petal-related features are more informative than sepal-
related ones for distinguishing species.
In the original data space (x1, x2, x3), data points are scattered and have varying variances across
dimensions. PCA identifies the directions of maximum variance in the data, known as PC1 and PC2,
which form the new axes in the principal component space. The data is then projected onto these new
axes, effectively reducing dimensionality while preserving the most informative aspects of the data.
This transformation helps in viewing and analyzing the data more effectively, especially for
visualization or further processing.
Additional Considerations before Using PCA
Effect Of Variable Magnitude On PCA Models
Selecting the Number of Clusters
1) ELBOW METHOD
First we need to decide how many clusters (k) we should use.
How It Works:
2)Silhouette Score
3)GAP STATISTICS
In K-means, choosing the right number of clusters is important because it can
greatly impact the results. The Gap Statistic helps solve this problem by
comparing how much the data within each cluster varies for different numbers of
clusters (k).
Gap Statistic compares how well the clusters formed from your actual data stand
out against what you would expect if the data were randomly distributed. In
other words, it looks at how tightly packed your real clusters are compared to
clusters created from random data. This helps us understand if the patterns we
see in our data are meaningful or just due to chance
The optimal number of clusters is the k where the gap between actual and reference is the largest.`
Hierarchical Clustering
Dendogram-For Hierarchical Clustering
(THE KOHONEN SOM ALGORITHM)
Visualising Kohonen Maps(As Clusters)
Steps in K-Means Algorithm
Similarity Between Kohonen SOM & K-Means Algorithm