Experiment No.

Objective: To perform dimensionality reduction operation using PCA for Houses Data Set.

Theory:
Principal Component Analysis (PCA) is a dimensionality reduction technique commonly
used in data analysis and machine learning. It aims to transform the original variables into a
new set of uncorrelated variables, known as principal components, while retaining as much of
the variance in the data as possible. Here's an overview of the theory behind PCA:

Key Concepts:

1. Covariance Matrix:
o PCA is based on the covariance matrix of the original data. The covariance matrix
summarizes the relationships between different variables. It helps identify the strength and
direction of linear relationships.

2. Eigenvalues and Eigenvectors:

o PCA involves finding the eigenvalues and eigenvectors of the covariance matrix.
Eigenvectors are the directions along which the data varies the most, and eigenvalues
represent the magnitude of the variance in those directions.

3. Principal Components:
o Principal components are linear combinations of the original variables and are obtained by
multiplying the original data by the matrix of eigenvectors. The first principal component has
the highest variance,and subsequent components have decreasing variance.

4. Explained Variance:
o Each principal component explains a certain proportion of the total variance in the data.
The cumulative explained variance is used to assess how much information is retained as we
consider more principal components.

Steps in PCA:

1. Standardization:
o Standardize the original data to ensure that all variables have a mean of 0 and a standard
deviation of 1. This step is crucial for PCA since it is sensitive to the scale of the variables.

2. Covariance Matrix Calculation:

o Calculate the covariance matrix of the standardized data.

3. Eigenvalue and Eigenvector Computation:

o Find the eigenvalues and eigenvectors of the covariance matrix.
4. Selection of Principal Components:
o Sort the eigenvectors based on their corresponding eigenvalues in decreasing order. Select
the top kk eigenvectors to form the matrix WW (where kk is the desired number of
dimensions).

5. Projection onto Lower-Dimensional Space:

o Multiply the original standardized data by the matrix WW to obtain the transformed data in
the lower-dimensional space.

Applications of PCA:

1. Dimensionality Reduction:
o PCA is primarily used for reducing the number of variables in a dataset while retaining
most of the original information.

2. Data Visualization:
o PCA is often employed for visualizing high-dimensional data in two or three dimensions.

3. Noise Reduction:
o PCA can help reduce the impact of noise in data by focusing on the directions with the most
variance.

4. Feature Extraction:
o PCA can be used for feature extraction, identifying the most important features in the data.

5. Compression and Reconstruction:

o PCA can be used for compressing data and reconstructing it with minimal [Link] is a
powerful tool for understanding and simplifying complex datasets, making it widely applied
in various fields such as image processing, signal processing, finance, and biology.

Code:

# Install and load necessary libraries

if (!requireNamespace("MASS", quietly = TRUE)) {
[Link]("MASS")
}
library(MASS)
# Load the Houses dataset
data("Houses")
# Display the structure of the original dataset
cat("Original Houses Dataset:\n")
str(Houses)
# Separate features and target variable
features <- Houses[, -1]
# Exclude the first column (Price)
target <- Houses$Price
# Perform PCA
pca_result <- prcomp(features, center = TRUE, scale. = TRUE)
# Display the summary of PCA
53
summary(pca_result)
# Scree plot to visualize the variance explained by each principal component
screeplot(pca_result, type = "line", main = "Scree Plot")
# Biplot to visualize the relationship between original features and principal
components
biplot(pca_result, scale = 0)
# Extract transformed data with reduced dimensionality
reduced_data <- [Link](predict(pca_result, newdata = features))
# Display the transformed data with reduced dimensionality
cat("\nTransformed Data with Reduced Dimensionality:\n")
print(reduced_data)

Output: