Principal Component Analysis (PCA)

Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Principal Component Analysis, Recommender

Systems, and Beyond

Phillip Yam
Department of Statistics and Data Science, Chinese University of Hong Kong

Chapter 9: Principal Component Analysis and Recommender Systems

Chen, Cheung and Yam (2024). Financial Data Analytics with Machine Learning,
Optimization and Statistics. Wiley Finance Series.

1 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Table of Contents

1 Principal Component Analysis (PCA)

2 Immunization

3 Investment Strategy via PCA

4 Facial Recognition

5 Recommender System

2 / 30
Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Principal Component Analysis (PCA)

2 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Principal Component Analysis (PCA)

Principal Component Analysis (PCA for short) is a classical multivariate
statistical technique for a dimension reduction of an underlying set of random
variables.
It is used to find linear combinations of original variables x1 , . . . , xp such
that the information in the original data is “essentially” preserved even
after the reduction.
For i = 1, . . . , p,
yi = αi1 x1 + . . . + αip xp = αi⊤ x

where
the loadings of the i-th PC αi = (αi1 , . . . , αip )⊤ ,
x = (x1 , . . . , xp )⊤ .
αi′ s are found by applying Spectral Decomposition to the
covariance/correlation matrix of x.
λi : i-th largest eigenvalue, λi = Var(yi ),
αi : corresponding eigenvector of λi .
3 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

US Zero-Coupon Rates
US semi-annualized zero-coupon rates with maturities of 1M (months) to
15Y (years), monthly data over the period of 1944 to 1992
All 13 variables are highly correlated with each other

It would be good if we can use few variables to represent all the

variables...
⇒ We can apply PCA for dimension reduction!
4 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

US Zero-Coupon Rates
PC loadings and scree plot (plot of explained variance):

Explained variance:
1st PC: 97.88%, 2nd PC: 1.83%, 3rd PC: 0.19%
First 3 PCs preserve 99% of all the information.
Dimension reduction: drop the remaining PCs as they only preserve very
small amount of information.
5 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

US Zero-Coupon Rates

1st PC: 2nd PC: 3rd PC:

The loadings are almost The loadings increase with The loadings first decrease,
constant. maturity. then increase with maturity.
Interpreted as parallel shift. Interpreted as tilt component. Interpreted as curvature.
This can be referred to some This agrees with the liquidity Very short term bonds are
macroeconomic event having preference theory which welcomed by frequent traders
same impact on bonds with states that investors prefer
different maturity. more liquid assets Very long term bonds are
welcomed by insurance and
Additional premiums are paid reinsurance companies
for bonds with longer
maturity

6 / 30
Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Immunization

6 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Immunization for Interest Rate Risks

Consider a fund that consists of assets and liabilities.

At force of interest δ and time 0:
X A
−δti
Present value of the assets PA (δ; 0) = cA
i e ,
i
X L
−δti
Present value of the liabilities PL (δ; 0) = cL
i e
i

where cA
i (resp. cL
i )
and tA
(resp.
i tL
i )
are the amount and the time of
the i-th cash flow corresponding to the assets (resp. liabilities).
A change in the force of interest might result in a decrease of portfolio
value:
It would be good if the portfolio does not suffer from any adverse effect
due to a small change in the force of interest...

7 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Immunization for Interest Rate Risks

In order to be immunized against a small change ε in the force of
interest (Redington immunization), we need

PA (δ; 0) = PL (δ; 0),

PA (δ + ϵ; 0) ≥ PL (δ + ϵ; 0).

Using the Taylor series expansion, above conditions require

PA′ (δ; 0) = PL′ (δ; 0),

PA′′ (δ; 0) ≥ PL′′ (δ; 0).

To archive Redington immunization, the following conditions have to be

satisfied:
1 PA (δ; 0) = PL (δ; 0);
2 PA′ (δ; 0) = PL′ (δ; 0);
3 PA′′ (δ; 0) ≥ PL′′ (δ; 0).

8 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Immunization for Interest Rate Risks

The Macaulay duration of the assets is defined as
P A A −δtA
dPA /PA PA′ (δ; 0) i ti ci e
i
DA (δ) := − =− = P
dδ PA (δ; 0) A −δtA
i ci e
i

which is the average payment time of cash flows weighted by the

corresponding discounted cash flow.
It is regarded as:
The sensitivity of the price with respect to changes of the interest rate;
The effective lifespan of a project.
The convexity of the assets is defined as
A
A 2 A −δti
PA′′ (δ; 0)
P
i (ti ) ci e
= P A −δtA
PA (δ; 0)
i ci e
i

The Macaulay duration and the convexity of the liabilities are defined
similarly with A replaced by L.
9 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Immunization for Interest Rate Risks

To archive Redington immunization,

the Macaulay duration of the assets should be equal to that of the
liabilities;
the convexity of the assets should be greater than that of the
liabilities.
By denoting the net present value of the portfolio at time t as
P (δ; t) := PA (δ; t) − PL (δ; t), the conditions for Redington immunization
can be further rewriten as
1 P (δ; 0) = 0;
2 P ′ (δ; 0) = 0;
3 P ′′ (δ; 0) ≥ 0.

10 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Immunization for Interest Rate Risks

Example:
One liability cash outflow of LT at time T .
Try to hold two assets to archive immunization:
Asset 1: provides cash inflow of A at time T − a (a ∈ (0, T ]),
Asset 2: provides cash inflow of B at time T + b (b > 0)
If we know any two of A, B, a, b, how to find out the other two?
Using T rather than 0 as the comparison date, Solve
(
P (δ; T ) = Aeaδ + Be−bδ − LT = 0,
P ′ (δ; T ) = aAeaδ − bBe−bδ = 0

Is P ′′ (δ; T ) >= 0?

P ′′ (δ; T ) = a2 Aeaδ + b2 Be−bδ > 0

Hence, the portfolio that satisfies above equation set is immunized

against a small change ε in the force of interest.
11 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Full Immunization
What about ε that is not small?
By solving above set of equation, we have
Be−bδ = ab Aeaδ = a+b a
LT .
Thus,
P (δ + ϵ; T ) = Aea(δ+ϵ) + Be−b(δ+ϵ) − LT
= Aea(δ+ϵ) + Be−b(δ+ϵ) − (Aeaδ + Be−bδ )
 a  a 
= Aeaδ eaε + e−bε − 1 +
b b
Denote f (ε) := eaε + ab e−bε − 1 + ab . By taking its first derivative, we

have f ′ (ε) = a(eaε − e−bε ), which

f ′ (ε) < 0 when ε < 0,
f ′ (ε) > 0 when ε > 0,
f ′ (ε) = 0 when ε = 0
Therefore, f (ε) takes its global minimum at ε = 0.
⇒ P (δ + ϵ; T ) ≥ P (δ; T ) for any ε ∈ R
The portfolio strategy in this example has also archived full
immunization.
12 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Portfolio Immunization
Suppose there are n assets in the market with the change in growth rate
∆Rti = Rt+1
i
− Rti for i = 1, . . . , n.
We can extract the latent risk factors by applying PCA.
We model the change in growth rate ∆Rti = Rt+1
i
− Rti by using a linear
k-factor model:
k
X
∆Rti = wli Ytl + ϵit , i = 1, . . . , n.
l=1

where
YTl is the value of the l-th principal component (risk factors)
of the n assets at time t for l = 1, . . . , k;
wli is the sensitivity in growth of the i-th asset with respect to
the l-th risk factor;
ϵit is the residual, or miscellaneous, risk factor.

13 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Portfolio Immunization

Meanwhile, let Πt be the value of the portfolio at time t. We expect to

use (at least) k other (structural) securities to hedge out most of the
risks. Hence, the value of the combined portfolio at time t is given by
k
X
Π̄t = Πt + ϕjt Stj
j=1

where
Stj is the respective prices of those other k securities at time t;
ϕjt is the units of the j-th security we hold for hedging

14 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Portfolio Immunization

Denoting the change in value of the combined portfolio from t to t + 1 by

∆Π̄t , we have
 
n k j
X ∂Π t
X j ∂S t  ∆Rti
∆Π̄t ≈  + ϕt
i=1
∂Rti j=1
∂Rti
  !
n k j k
X ∂Π t
X j ∂S X
= 
i
+ ϕt t
i
 i l
wl Yt  + miscellaneous error involving ϵit
i=1
∂R t j=1
∂R t l=1
  
k n k
X j ∂S j
X X ∂Π t
=  wli  + ϕt t  l
Yt + miscellaneous error involving ϵit
l=1 i=1
∂Rti j=1
∂Rti

To (delta) hedge out the risky position (be immunized against the change
in Yt1 , . . . , Ytk ), we want to ensure that all coefficients of Ytl vanish for all
t.

15 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Portfolio Immunization

Hence, ϕjt , j = 1, . . . , k, should be chosen by solving the system of linear

equations:
n k
!
X i ∂Πt X j ∂Stj
wl + ϕ = 0, for l = 1, . . . , k
i=1
∂Rti j=1 t ∂Rti

or equivalently,
Pt + St Φt = 0
where
Φt = (ϕ1t , . . . , ϕkt )⊤ ,
P ⊤
n i ∂Πt Pn i ∂Πt
Pt = i=1 w1 ∂Ri , . . . , i=1 wk ∂Ri ,
 t t
j
Pn i ∂St
St = i=1 wl ∂Ri
t
lj

16 / 30
Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Investment Strategy via PCA

16 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Investment Strategy

During first half of the year in 2020, there was a global stock decline due
to the outbreak of COVID-19.
Data: S&P 500 component stocks, early 2020
We can apply PCA to the US stock returns x1 , . . . , xn :
The 1st PC y1 is interpreted as major market factor
The 1st PC explains most of the variance of x1 , . . . , xn .
The return of security i ≈ hi1 y1 if all other PCs are counted out.

17 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Investment Strategy (Cont.)

Extract the loadings of the 1st PC:
|α |
PCA portfolio: weight of the i-th security wi = P500 1i
j=1 |α1j |
Approximate the returns of S&P 500
Winning portfolio: long the top 10 securities (equal weight) with
highest corresponding first PCA loadings.

Best-performing stocks during the decline are daily consumer-product

related.
Sensible result. Customers still need necessity goods during the
quarantine period.
18 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Investment Strategy (Cont.)

Result:

Outperformance against S&P500 index of the PCA weighted portfolio of

top 10 securities with the largest loadings hi1 during the first half year of
2020.

19 / 30
Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Facial Recognition

19 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Facial Recognition
In facial recognition, we determine the identity of an individual from a set of
images, each having a common height h and width w.

IU, Ji-eun LEE. Cropping Courtesy: Courtesy:

MediaPipe MediaPipe

Figure 1: Facial recognition for the actress IU, a contemporary popular

South Korean singer–songwriter and actress. Source: Sipa USA.

20 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Facial Recognition with PCA

The observation xi , for i = 1, . . . , n, is the pixel values of the i-th flattened face
image.
The features xj , for j = 1, . . . , h · w, are locally highly correlated, because the
neighborhoods of a pixel xj often share a similar color.
⇒ We can apply PCA for dimension reduction, selecting key features.
In the context of facial recognition, for classification, we compute the Euclidean
distance of a query face image x∗ against the database x1 , . . . , xn with
respective labels y1 , . . . , yn .
Suppose that the first m PCs are selected with the corresponding loadings
H (1) = (h1 , . . . , hm ) ∈ R(h·w)×m , the predicted label of that query face image
is given by y∗ = yk , where
m
X
k = arg min d((H (1) )⊤ xi , (H (1) )⊤ x∗ ) = arg min (h⊤ 2
j (xi − x∗ )) .
i=1,...,n i=1,...,n j=1

21 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Facial Recognition with AT&T Database of Faces

40 individuals taken between April 1992 and April 1994.

10 face images of size 112 × 92(= h × w) in different scenarios per
individual;
varying lighting, having closed eyes, smiling, and wearing
glasses.
Testing set: first scenario; Traning set: remaining 9 scenarios forming a
(112 · 92) × (9 · 40) = 10, 304 × 360 matrix X.
Notice that XX ⊤ is a singular 10, 304 × 10, 304 matrix, but we know
that the largest 360 (non-zero) eigenvalues of XX ⊤ and X ⊤ X are the
same.
⇒ We can apply PCA on X ⊤ X instead and select the first m PCs such
that 90% of all information are preserved.

22 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Facial Recognition with AT&T Database of Faces

23 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Facial Recognition with AT&T Database of Faces

Only one mismatch was found: The (34 + 1 =) 35-th test image was of a
lady but the classifier returns a male.
The mismatch has a distance of 78.546, which is the largest
among all pairs, indicating a relatively low confidence in the
classification.
In practice, we can set a threshold τ > 0, let say 65, such that we regard
the matching process of that query face image with the database as a
failure whenever the distance is larger than τ .

24 / 30
Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Recommender System

24 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Recommender System
A recommender system, commonly encountered in Facebook, Amazon,
and Netflix, helps to predict the ”rating” or ”preference” that a user
would give to an item.
m subscribers and n items.
The ratings are stored in a sparse rating matrix R = (ri,j )m,n .
Unrated entries are labelled as 0.
Goal: predict unrated entries in R.

Correlation-based method: predict ratings with a similarity measure.

Common choice: cosine similarity
x·y
cos(x, y) =
∥x∥∥y∥
25 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

User-based Recommender System

User-based method:
predict the rating of user i′ to item j ′ (i.e. ri′ ,j ′ ) by aggregating the
ratings of similar users of u′ .
Denote the rating vector of user i (i-th row of R) as ri , i = 1, . . . , m.
1 Compute si′ ,i = cos(ri′ , ri ) for all other users i that have rated item j ′ .

(N )
2 Find the top-N most similar (highest si′ ,i ) users i ∈ Ui′ .
3 Aggregate (e.g. take average) the their ratings of item j ′ as prediction:

1 X
r̂i′ ,j ′ = ri,j ′ .
N (N )
i∈U ′
i

26 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Singular Value Decomposition (SVD) for Recommender

System
Latent-factor method (matrix factorization):
We try to express R as the product of the user matrix P = (pi,k )m,p and the
item matrix Q = (qj,k )n,p .
Each user and movie, it is represented by a vector of ”latent/hidden”
features with dimension p.
Usually, we would choose p ranged from 20 to 500 (much less than m and
n).
For a user i’s rating of movie j, it is modelled as r̂i,j = pk=1 pi,k qj,k
P

In matrix form, R̂ = PQ⊤

27 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Singular Value Decomposition (SVD) for Recommender

System
Mathematically, we have to solve for P, Q such that
J(pi,k , qj,k ; 1 ≤ i ≤ m, 1 ≤ j ≤ n, 1 ≤ k ≤ p)
p
m X n
!
X X
= ri.j − pi.k qj,k
i=1 j=1 k=1
m X
X n  
= ri.j − p⊤
i qj
i=1 j=1

= tr((R − R̂)⊤ (R − R̂))

= tr(R⊤ R) − 2tr(PQ⊤ R) + tr(PQ⊤ QP⊤ )
is minimized.
First order condition: equate ∇P J and ∇Q J with zero
(
RQ = PQ⊤ Q
R⊤ P = QP⊤ P
How to solve? 28 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Singular Value Decomposition (SVD) for Recommender

System
Singular Value Decomposition (SVD)
R = U ΣV ⊤

where
U ∈ Rm×m is the matrix of left singular vectors,
V ∈ Rn×n is the matrix of right singular vectors,
Σ ∈ Rm×n is a diagonal matrix with diagonal entries filled
with singular values.
For recommender system, we use only p largest singular values in Σ:
P = U Σ1 , Q = V Σ⊤
2

where
   
D1 D2
Σ1 = , Σ⊤
2 = , D1 D2⊤ = diag(σ1 , . . . , σp )
0 0

29 / 30
 Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Singular Value Decomposition (SVD) for Recommender

System
Example: Movie Recommendation - MovieLens
m = 943 users, n = 1664 movies, 100000 rating records

30 / 30
Principal Component Analysis (PCA)
Immunization
Investment Strategy via PCA
Facial Recognition
Recommender System

Thank you!

Chapter 9

30 / 30