Scribd
Upload
22 views2 pages

PCA and LDA Tutorial for Data Scientists

The tutorial focuses on Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) for classifying datasets in R2. It includes problems requiring the identification of principal components and the evaluation of classification capabilities based on projections. Additionally, it provides a design matrix for PCA computation, emphasizing the necessity of centering data and calculating eigenvectors and eigenvalues.

Uploaded by

karandeepsinghsaluja963
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
22 views2 pages

PCA and LDA Tutorial for Data Scientists

The tutorial focuses on Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) for classifying datasets in R2. It includes problems requiring the identification of principal components and the evaluation of classification capabilities based on projections. Additionally, it provides a design matrix for PCA computation, emphasizing the necessity of centering data and calculating eigenvectors and eigenvalues.

Uploaded by

karandeepsinghsaluja963
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

DATA7703 – Machine Learning for Data Scientists S2 - 2025

Tutorial – Week 4
PCA & LDA

The below tutorial problems are meant to be solved using pen and paper.
1. Principal Component Analysis

In the following plots, a train set of data points X belonging to two classes on R2
are given, where the original features are the coordinates (x, y). For each, answer the
following questions:

(a) Draw all the principal components.

(b) Can we correctly classify this dataset by using a threshold function after
projecting onto one of the principal components? If so, which principal component
should we project onto? If not, explain in 1–2 sentences why it is not possible.

Dataset 1 Dataset 2

(left) use orange eigenvector (right) use green eigenvector


2. Consider the following design matrix, representing four sample points 𝑋𝑋𝑖𝑖 ∈ 𝑅𝑅 2.

4 1
𝑋𝑋 = �2 3�
5 4
1 0

We want to represent the data in only one dimension, so we turn to principal components analysis
(PCA).

(a) Compute the unit-length principal component directions of X, and state which one the PCA
algorithm would choose if you request just one principal component. Please provide an exact answer,
without approximation. (You will need to use the square root symbol.)
Show your work!

Answer:

We center X, by subtracting the mean

μ = [3 2]
yielding
1 −1
𝑋𝑋 − μ = �−1 1 �
2 2
−2 −2
10 6
Then (X- μ)T(X- μ) = � � (Divide by 3 if you want the sample covariance matrix. But we don’t
6 10
care about the magnitude.)
1 1 1 −1
Its eigenvectors are [ ] with eigenvalue 16 and [ ] with eigenvalue 4.
√2 √2 √2 √2

The former eigenvector is chosen. (Negated versions of these vectors is also correct.)

You might also like