Introduction to PCA
UN SUP ERVISED L EARN IN G IN R
Hank Roark
Senior Data Scientist at Boeing
Two methods of clustering
Two methods of clustering - nding groups of homogeneous items
Next up, dimensionality reduction
Find structure in features
Aid in visualization
UNSUPERVISED LEARNING IN R
Dimensionality reduction
A popular method is principal component analysis (PCA)
Three goals when nding lower dimensional representation of features:
Find linear combination of variables to create principal components
Maintain most variance in the data
Principal components are uncorrelated (i.e. orthogonal to each other)
UNSUPERVISED LEARNING IN R
PCA intuition
UNSUPERVISED LEARNING IN R
PCA intuition
UNSUPERVISED LEARNING IN R
PCA intuition
UNSUPERVISED LEARNING IN R
Visualization of high dimensional data
UNSUPERVISED LEARNING IN R
Visualization
UNSUPERVISED LEARNING IN R
PCA in R
[Link] <- prcomp(x = iris[-5],
scale = FALSE,
center = TRUE)
summary([Link])
Importance of components:
PC1 PC2 PC3 PC4
Standard deviation 2.0563 0.49262 0.2797 0.15439
Proportion of Variance 0.9246 0.05307 0.0171 0.00521
Cumulative Proportion 0.9246 0.97769 0.9948 1.00000
UNSUPERVISED LEARNING IN R
Let's practice!
UN SUP ERVISED L EARN IN G IN R
Visualizing and
interpreting PCA
results
UN SUP ERVISED L EARN IN G IN R
Hank Roark
Senior Data Scientist at Boeing
Biplot
UNSUPERVISED LEARNING IN R
Scree plot
UNSUPERVISED LEARNING IN R
Biplots in R
# Creating a biplot
[Link] <- prcomp(x = iris[-5],
scale = FALSE,
center = TRUE)
biplot([Link])
UNSUPERVISED LEARNING IN R
Scree plots in R
# Getting proportion of variance for a scree plot
[Link] <- [Link]$sdev^2
pve <- [Link] / sum([Link])
# Plot variance explained for each principal component
plot(pve, xlab = "Principal Component",
ylab = "Proportion of Variance Explained",
ylim = c(0, 1), type = "b")
UNSUPERVISED LEARNING IN R
UNSUPERVISED LEARNING IN R
Let's practice!
UN SUP ERVISED L EARN IN G IN R
Practical issues with
PCA
UN SUP ERVISED L EARN IN G IN R
Hank Roark
Senior Data Scientist at Boeing
Practical issues with PCA
Scaling the data
Missing values:
Drop observations with missing values
Impute / estimate missing values
Categorical data:
Do not use categorical data features
Encode categorical features as numbers
UNSUPERVISED LEARNING IN R
mtcars dataset
data(mtcars)
head(mtcars)
mpg cyl disp hp drat wt qsec vs
Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0
Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0
Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1
Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1
Hornet Sportabout 18.7 8 360 175 3.15 3.440 17.02 0
Valiant 18.1 6 225 105 2.76 3.460 20.22 1
UNSUPERVISED LEARNING IN R
Scaling
# Means and standard deviations vary a lot
round(colMeans(mtcars), 2)
mpg cyl disp hp drat wt qsec vs
20.09 6.19 230.72 146.69 3.60 3.22 17.85 0.44
round(apply(mtcars, 2, sd), 2)
mpg cyl disp hp drat wt qsec vs
6.03 1.79 123.94 68.56 0.53 0.98 1.79 0.50
UNSUPERVISED LEARNING IN R
Importance of scaling data
UNSUPERVISED LEARNING IN R
Scaling and PCA in R
prcomp(x, center = TRUE, scale = FALSE)
UNSUPERVISED LEARNING IN R
Let's practice!
UN SUP ERVISED L EARN IN G IN R
Additional uses of
PCA and wrap-up
UN SUP ERVISED L EARN IN G IN R
Hank Roark
Senior Data Scientist at Boeing
Dimensionality reduction
UNSUPERVISED LEARNING IN R
Data visualization
UNSUPERVISED LEARNING IN R
Interpreting PCA results
UNSUPERVISED LEARNING IN R
Importance of data scaling
UNSUPERVISED LEARNING IN R
Up next
# URL to cancer dataset hosted on DataCamp servers
url <- "[Link]
# Download the data: [Link]
[Link] <- [Link](url)
[Link][1:6, 1:5]
radius_mean texture_mean perimeter_mean area_mean smoothness_mean
842302 17.99 10.38 122.80 1001.0 0.11840
842517 20.57 17.77 132.90 1326.0 0.08474
84300903 19.69 21.25 130.00 1203.0 0.10960
84348301 11.42 20.38 77.58 386.1 0.14250
84358402 20.29 14.34 135.10 1297.0 0.10030
843786 12.45 15.70 82.57 477.1 0.12780
UNSUPERVISED LEARNING IN R
Let's practice!
UN SUP ERVISED L EARN IN G IN R