FACTOR ANALYSIS

27 April 2008 Prepared by C Y Nimkar 1

 Introduction
• Suppose we ask a respondent who is likely to buy
a 4-wheeler to rate the importance that he would
give to various aspects of a 4-wheeler like:
– Mileage
– Price
– smooth ride
– cost of spare parts
– servicing locations
– Cooling effect
– Leg space and so on
27 April 2008 Prepared by C Y Nimkar 2
 Introduction
• Suppose he gives higher rating to following
attributes:
– Leg space
– Smooth ride
– Cooling effect
– Interiors
What is he looking for?
He is looking for comfort
27 April 2008 Prepared by C Y Nimkar 3
 Introduction
Similarly, if the respondent gives high scores on
following attributes:
– Price of car
– Mileage
– Price of spare parts
– Interest on loan
What does he want?
He wants economy
27 April 2008 Prepared by C Y Nimkar 4
 Introduction
• Thus ‘comfort’, ‘economy’ are not single
measurable entities but complex constructs
derived from many inter-related variables
• Such complex constructs are called Factors.
Identifying such factors simplifies understanding a
complex phenomenon
• Factor analysis is a tool that is used to identify
factors from many inter-related variables.
Therefore, it is also called as data reduction
technique.

27 April 2008 Prepared by C Y Nimkar 5

 Factor analysis model

27 April 2008 Prepared by C Y Nimkar 6

Ratings on 7 point scale
Correlation between mileage & price ratings = 0.7739
Mileage Price
7 6
Price
2 3
5 4
6 5
4 5
6 7
0.7739
7 6
4 3
6 7 There is 77.39%
7 6 tendency
2 4 to give similar rating
Mileage on mileage and
price
27 April 2008 Prepared by C Y Nimkar 7
 • His ism

Leg space (X1) = w1 (comfort) + w2 (economy) + U1

Leg space =

27 April 2008 Prepared by C Y Nimkar 8

 Factor analysis model
Leg space (X1) = w1 (comfort) + w2 (economy) + U1
• w1 is the index of relation between leg space &
comfort and w2 between leg space & economy.
• Since leg space is more related to comfort than
economy, w1 will have higher value than w2.
• U1 is that part of leg space variable not expressed
by factors ‘comfort’ & ‘economy’. It is uniqueness
of the variable

27 April 2008 Prepared by C Y Nimkar 9

 Factor analysis model
In general, original variable can be expressed
as a linear combination of factors as below:
Xi = wi1 (F1) + wi2 (F2)+………+ wip (Fp)+Ui
wi1, wi2, … are called factor loadings & F1,
F2,… are called common factors or simply
factors.
Ui is the part of Xi not explained by factors.

27 April 2008 Prepared by C Y Nimkar 10

 Steps in Factor analysis

27 April 2008 Prepared by C Y Nimkar 11

• Step 1: Prepare statements:
– Respondents are given statements related to the
subject & response is obtained on Graphic
rating scale. Often ‘Likert scale’ is used.
– The responses are converted into scores
– Factor analysis is run on scores

27 April 2008 Prepared by C Y Nimkar 12

• Step 2: To check goodness of fit
– Following are three outputs that indicate
appropriateness of model:
• Correlation matrix
• Bartlett’s test of sphericity
• Kaiser-Meyre-Olkin(KMO) statistic

27 April 2008 Prepared by C Y Nimkar 13

 Correlation matrix

Mileage Price Leg space Smooth ride

Mileage 1.00 0.77 0.15 0.08
Price 0.77 1.00 0.21 0.19
Leg space 0.15 0.21 1.00 0.85
Smooth ride 0.08 0.19 0.85 1.00

• Correlation matrix:
– High non-diagonal values in correlation
matrix show variables in row & column
are related
– If such values exist, factor analysis will
be appropriate for the data.

27 April 2008 Prepared by C Y Nimkar 14

 Bartlett’s test

– Bartlett’s test of sphericity is used to test null

hypothesis that correlation matrix is Identity matrix.

Correlation matrix Identity matrix

Mileage Price Leg Smooth Mileage Price Leg Smooth

space ride space ride
Mileage 1.00 0.77 0.15 0.08 Mileage 1.00 0 0 0
Price 0.77 1.00 0.21 0.19 Price 0 1.00 0 0
Leg
space
Smooth
0.15

0.08
0.21

0.19
1.00

0.85
0.85

1.00
= Leg
space
Smooth
0

0
0

0
1.00

0 1.00
0

ride ride

27 April 2008 Prepared by C Y Nimkar 15

 Kaiser-Meyer-Olkin statistic

– Kaiser-Meyer-Olkin (KMO) value also tells us

whether factor analysis is appropriate for data.
– If KMO >= 0.50, factor analysis is appropriate
for the data

27 April 2008 Prepared by C Y Nimkar 16

• Step 3: Interpret factor loadings
– Let us consider following factor loading matrix
(matrix of w’s):

Factor loading matrix

Variables Factor 1 Factor 2

Mileage 0.8045 -0.2578
Price 0.7245 0.2354
Leg space -0.2585 0.9541
Smooth ride 0.1258 0.8521

27 April 2008 Prepared by C Y Nimkar 17

 Factor loading matrix

Variables Factor 1 Factor 2

Mileage 0.8045 -0.2578

Price 0.7245 0.2354
Leg space -0.2585 0.7541
Smooth ride 0.1258 0.6521

Here the model will be:

Mileage(X1) = 0.8045 F1 – 0.2578 F2+U1
Price(X2) = 0.7245 F1 + 0.2354 F2+U2
Leg space(X3) = -0.2585 F1 + 0.7541 F2+U3
Smooth ride(X4) = 0.1258 F1 + 0.6521 F2+U4

27 April 2008 Prepared by C Y Nimkar 18

• Information captured by a factor is called its eigenvalue.
• By default, number of factors = number of variables
• It is computed as sum of squares of factor loadings on the factor.
• Eigenvalue for first factor is 1.2548 and second factor is 1.1158
• Eigenvalue for third factor is 0.9542 and fourth factor is 0.6752
(not shown)

Variable Factor 1 Factor 2

Mileage 0.8045 -0.2578
Price 0.7245 0.2354
Leg space -0.2585 0.7541
Smooth ride 0.1258 0.6521

Eigenvalue (0.8045)2+(0.7245)2+(-0.2585)2 (-0.2578)2 + (0.2354)2 +

(Sum of squares) +(0.1258)2= 1.2548 (0.7541)2+(0.6521)2 = 1.1158

27 April 2008 Prepared by C Y Nimkar 19

 • Step 4: To decide no. of factors:

Information given Information captured

Variable No. of units of information Factor No. of units of information
given captured
Mileage 1.00 Factor 1 1.2548
Price 1.00 Factor 2 1.1158
Leg space 1.00
Factor 3 0.9542
Smooth ride 1.00
Factor 4 0.6752
TOTAL 4.00
TOTAL 4.00

1.3 1.3

1.2 1.2

1.1 1.1

1.0 1.0
Value Information given

.9
.9

.8 Value Eigenvalue .8

.7
.7

.6
.6
Mileage Price Leg space Smooth ride
Mileage Price Leg space Smooth ride

Variables
Variables

27 April 2008 Prepared by C Y Nimkar 20

• Step 4: To decide no. of factors:
– Factors with eigenvalue more than 1 are only
considered
– In last table eigenvalues of factor 3 and factor 4 are
less than 1, therefore, not shown

27 April 2008 Prepared by C Y Nimkar 21

 – Once number of factors are decided, find information captured by
them from each variable
– This is called communality of each variable
– It is sum of squares of factor loadings of the variable

Variable Factor 1 Factor 2 Communality

Mileage 0.8045 -0.2578 (0.8045)2 + (-0.2578)2 = 0.7137
Price 0.7245 0.2354 (0.7245)2 + (0.2354)2 = 0.5803
Leg space -0.2585 0.7541 (-0.2585)2 + (0.7541)2 = 0.6355
Smooth ride 0.1258 0.6521 (0.1258)2 + (0.6521)2 = 0.4411

• Communality of X1 is 0.7137. It means out of total information of 1

unit given by X1, 0.7137 is captured by both factors . Remaining
0.2863 could not be captured.
• It is U1in the model

27 April 2008 Prepared by C Y Nimkar 22

 Variable Factor 1 Factor 2

Mileage 0.8045 -0.2578

Price 0.7245 0.2354
Leg space -0.2585 0.7541
Smooth ride 0.1258 0.6521

• Mileage and price can be associated with Factor 1

• Leg space and smooth ride can be with Factor 2

27 April 2008 Prepared by C Y Nimkar 23

 Factor loading matrix 1 Factor loading matrix 2
Variable Factor 1 Factor 2 Variable Factor 1 Factor 2

Mileage 0.8045 -0.2578 Mileage 0.3189 0.2758

Price 0.7245 0.2354 Price 0.4245 0.5354
Leg space -0.2585 0.7541 Leg space -0.4085 0.4841
Smooth 0.1258 0.6521 Smooth -0.5258 0.4921
ride ride

• Ideally factor loadings should be high only on one factor

• If not, it can be achieved by factor rotation

27 April 2008 Prepared by C Y Nimkar 24

• Steps 5: Rotate axis:
– To understand factor rotation, let us see the
graph in next slide

27 April 2008 Prepared by C Y Nimkar 25

Variable Factor 1 Factor 2
Mileage 0.3189 0.2758 .6
Price 0.4245 0.5354
Leg space -0.4085 0.4841 Price
Smooth -0.5258 0.4921
ride .5 Smooth ride
Leg space

New Factor 1
.4

New Factor 2

.3
Mileage
Factor 2

.2
-.6 -.4 -.2 -.0 .2 .4 .6

Factor 1

27 April 2008 Prepared by C Y Nimkar 26

 – By rotating axis, points have come closer to
only one axis.
– Such factor loadings are easy to interpret
– ‘Varimax’ rotation of axis is preferred method
of rotation

27 April 2008 Prepared by C Y Nimkar 27

• Step 6: Name factors:
– By combining variables with high factor loadings,
irrespective of sign, will give us factors
– Based on variables that get combined, we name the
factor. Naming the factor is everybody’s judgment
– We can give names ‘economy’ to factor 1 and
‘comfort’ to factor 2

27 April 2008 Prepared by C Y Nimkar 28

 END OF
FACTOR ANALYSIS

27 April 2008 Prepared by C Y Nimkar 29