See discussions, stats, and author profiles for this publication at: [Link]

net/publication/344898204

Factor Analysis

Chapter · January 2017

DOI: 10.4018/978-1-68318-016-6.ch007

CITATION READS

1 1,740

2 authors:

Rui Sarmento Vera Costa

79 PUBLICATIONS 222 CITATIONS

University of Porto
24 PUBLICATIONS 128 CITATIONS
SEE PROFILE
SEE PROFILE

All content following this page was uploaded by Rui Sarmento on 28 April 2021.

The user has requested enhancement of the downloaded file.

 116

Factor Analysis
INTRODUCTION

Factor analysis is a statistical method used to describe variability among observed, correlated variables.
The goal of performing factor analysis is to search for some unobserved variables called factors. This
analysis might lead, for example, to the conclusion that it is possible that three unobserved latent variables
are reflected in the variations of seven observed variables. The observed variables are modeled as linear
combinations of the possible factors, added the error quantification of this approximation. This added
information about the interaction of observed variables could be used for further analysis of the
importance of each variable in the context of the dataset.

Factor analysis is used in many areas of statistical analysis like, for example, marketing, social sciences,
psychology and other situations where a reduction of a large set of variables is adequate to the study being
provided. This way, some observed variables are substituted by a set of latent variables in a lower
amount, and that, therefore, represent the data in a summarized fashion.

Factor analysis started by being developed before the appearance of modern computers. This beginning of
the method was named exploratory factor analysis (EFA). Other variations of factor analysis (for
example, confirmatory factor analysis - CFA) will not be explored in this book. Thus, an example of a
factorial analysis is presented below.

Example
Imagine a Ph.D. Supervisor wants to test the hypothesis there are two kinds of students. A student that
"procrastinates" his studies, and the student that does "not procrastinate", neither of which is an observed
variable. Thus, the supervisor only has access to the grades of the student in the several phases a Ph.D.
has. Suppose there are ten stages and the student is classified in all those stages. Additionally, the
supervisor has a database of 500 Ph.D. students. By choosing each student randomly from this vast
universe of students, imagine the grades as being random variables also. The supervisor hypothesis might
clarify that for each of the 10 Ph.D. grades, the score averaged over the group of all students who share
some common pair of values for procrastination and "not procrastinating" is some constant multiplied by
their level of procrastination plus another constant multiplied by their level of low inertia behaviour, i.e.,
it is a combination of those two "factors".
The numbers for a particular stage, by which the two kinds of behavior are multiplied to obtain the
expected score, are posited by the hypothesis to be the same for all procrastination level pairs and are
called "factor loading" for this subject. For example, the assumption may hold that the average student's
aptitude in the field of "State-of-the-Art writing" is {11 × the student's "procrastinating"} + {5 × the
student's "not procrastinating"}.
The numbers 11 and 5 are the factor loadings associated with the task of writing the State-of-the-Art
chapter. Other academic tasks may have different factor loadings.
Two students having similar degrees of procrastination and equal degrees of having low inertia may have
different aptitudes in State-of-the-Art writing because individual skills differ from average abilities. That
difference is called the "error" - a statistical term that means the amount by which an individual changes
from what is average for his or her levels of procrastination.

The observable data that go into factor analysis would be ten stage's scores of each of the 500 students, a
total of 5,000 numbers. The factor loadings and levels of the two kinds of inertia of each student should
be inferred from the data.
 117

THE FACTOR ANALYSIS MODEL

The scores of 𝑝 population variables, extracted from a population with mean’s vector 𝜇 and variance-
covariance matrix Σ, can be modeled by:
𝑥! = 𝜇! + 𝜆!! 𝑓! + 𝜆!" 𝑓! + ⋯ + 𝜆!! 𝑓! + 𝜂!
𝑥! = 𝜇! + 𝜆!" 𝑓! + 𝜆!! 𝑓! + ⋯ + 𝜆!! 𝑓! + 𝜂!
…
𝑥! = 𝜇! + 𝜆!! 𝑓! + 𝜆!! 𝑓! + ⋯ + 𝜆!" 𝑓! + 𝜂!
where 𝑓! are factor values (with 𝑚 < 𝑝), η! represent the 𝑝 specific factors and 𝜆!" represents the weight
of 𝑗 factor in the variable 𝑖 (factor loadings), that is, each 𝜆 !" measures the contribution of the 𝑗 common
factor in the variable 𝑖. Without loss of generality, and for convenience, 𝑥! variables can be centered and
reduced as 𝑧! = (𝑥! − 𝜇! )/𝜎! . Thus, the factor model can be written by:

𝑧! = 𝜆!! 𝑓! + 𝜆!! 𝑓! + ⋯ + 𝜆!" 𝑓! + 𝜂!        (𝑖 = 1, … , 𝑝)

Note that λ!" values are different depending on whether the analysis is done with the 𝑥! values (factor
weights) or 𝑧! (standardized factor weights). It must, therefore, be assumed that (Maroco, 2011):
• Common factors (𝑓! ) are independent (orthogonal) and equally distributed with mean 0 and
variance 1 (𝑘 = 1, … , 𝑚).
• Specific factors (𝜂! ) are independent and equally distributed with mean 0 and variance
𝜓! , 𝑗 = 1, … , 𝑝 .
• 𝑓! and 𝜂! are independent.

R VS. PYTHON
An example of factor analysis will be given in this chapter. Consider, for instance, the data from a survey
presented to the researchers. The goal of the surveys was to understand the researcher’s behavior. The
questionnaire presented to the researchers was explained in the chapter “Dataset” of this book. Since
different questions seem to evaluate the same characteristic, a division by factor is convenient.
Therefore, the first step is describing the variables, which allows discovering some irregularities such as
missing values or outliers.

In R
### Descriptive analysis of Q1 to Q10 variables

# Identification of the variables used in factor analysis

Code

survey<-data[, paste("Q", 1:10, sep="")]

# Descriptive analysis for each variable

summary(survey)
Q1 Q2 Q3 Q4 Q5
Min. :1.000 Min. :2.00 Min. :1.000 Min. :1.00 Min. :1.000
1st Qu.:3.000 1st Qu.:3.00 1st Qu.:3.000 1st Qu.:3.00 1st Qu.:3.000
Median :4.000 Median :4.00 Median :4.000 Median :3.00 Median :4.000
Mean :3.545 Mean :3.92 Mean :3.865 Mean :3.21 Mean :3.585
3rd Qu.:4.000 3rd Qu.:4.25 3rd Qu.:5.000 3rd Qu.:4.00 3rd Qu.:5.000
Output

Max. :5.000 Max. :5.00 Max. :5.000 Max. :5.00 Max. :5.000

Q6 Q7 Q8 Q9 Q10
Min. :1.00 Min. :1.00 Min. :1.0 Min. :1.000 Min. :1.000
1st Qu.:3.00 1st Qu.:3.75 1st Qu.:2.0 1st Qu.:3.000 1st Qu.:3.000
Median :4.00 Median :4.00 Median :3.0 Median :4.000 Median :4.000
Mean :3.38 Mean :4.01 Mean :3.1 Mean :3.825 Mean :3.905
3rd Qu.:4.00 3rd Qu.:5.00 3rd Qu.:4.0 3rd Qu.:5.000 3rd Qu.:5.000
Max. :5.00 Max. :5.00 Max. :5.0 Max. :5.000 Max. :5.000
Table 69
 118

In Python
### Descriptive analysis of the variables Q1 to Q10

# Read csv with survey data

import pandas as pd
Code

data = pd.read_csv('[Link]', sep=';')

survey_data = [Link][:,7:17]

# Summary of the variables

survey_data.describe()

Q1 Q2 Q3 Q4 Q5 Q6
count 200.000000 200.000000 200.00000 200.000000 200.00000 200.000000
mean 3.545000 3.920000 3.86500 3.210000 3.58500 3.380000
std 1.210496 0.834916 1.03544 0.959428 1.18736 1.082339
min 1.000000 2.000000 1.00000 1.000000 1.00000 1.000000
25% 3.000000 3.000000 3.00000 3.000000 3.00000 3.000000
50% 4.000000 4.000000 4.00000 3.000000 4.00000 4.000000
75% 4.000000 4.250000 5.00000 4.000000 5.00000 4.000000
Output

max 5.000000 5.000000 5.00000 5.000000 5.00000 5.000000

Q7 Q8 Q9 Q10
count 200.000000 200.00000 200.000000 200.000000
mean 4.010000 3.10000 3.825000 3.905000
std 0.951011 1.13421 1.188037 1.127979
min 1.000000 1.00000 1.000000 1.000000
25% 3.750000 2.00000 3.000000 3.000000
50% 4.000000 3.00000 4.000000 4.000000
75% 5.000000 4.00000 5.000000 5.000000
max 5.000000 5.00000 5.000000 5.000000
Table 70

The output above shows the non-existence of missing values. As the answers are represented on a Likert
scale, outliers should not exist, except when there are data errors. In this case, the errors should be
corrected, or the particular researcher’s data row should be eliminated. As it can be seen, all variables
vary between 1 and 5.

Depending on the variable type, different methods to obtain the correlation matrix should be used. In the
case of quantitative variables, Pearson correlations have satisfactory results. However, in the case of
nominal and ordinal variables, several authors (Marôco, 2011) advocate the use of other types of
correlations. This is regard to tetrachoric correlations for nominal variables and polychoric correlations
for ordinal data.
Although the polychoric's correlations present an excellent performance, the calculation and validation of
the associated assumptions require significant samples with 𝑛 > 1000. This condition limits the use of
polychoric correlations for smaller samples. Consequently, a highly clear analysis strategy for qualitative
data is the use of Cramer's V correlations for nominal variables, or Spearman correlations for ordinal
variables. Regarding that Q1 to Q10 variables are ordinal, in the example presented in this chapter,
Spearman's correlation will be the most suitable.

In R
### Correlation between variables Q1 to Q10
Code

correlation <- cor(survey, method="spearman")

correlation
 119

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10
Q1 1.0000 0.2217 0.2231 0.2567 0.4320 0.4041 0.2541 0.2718 0.5374 0.2845
Q2 0.2217 1.0000 0.3128 0.2653 0.2390 0.1934 0.1384 0.2253 0.0271 0.1507
Q3 0.2231 0.3128 1.0000 0.1542 0.2133 0.2597 0.2352 0.2377 0.0721 0.2486
Output

Q4 0.2567 0.2653 0.1542 1.0000 0.4236 0.1459 0.0817 0.3166 0.1489 0.2511
Q5 0.4320 0.2390 0.2133 0.4236 1.0000 0.3824 0.1575 0.4381 0.4419 0.2671
Q6 0.4041 0.1934 0.2597 0.1459 0.3824 1.0000 0.2281 0.2218 0.3081 0.3389
Q7 0.2541 0.1384 0.2352 0.0817 0.1575 0.2281 1.0000 0.1956 0.3257 0.2518
Q8 0.2718 0.2253 0.2377 0.3166 0.4381 0.2218 0.1956 1.0000 0.2751 0.2079
Q9 0.5374 0.0271 0.0721 0.1489 0.4419 0.3081 0.3257 0.2751 1.0000 0.2491
Q10 0.2845 0.1507 0.2486 0.2511 0.2671 0.3389 0.2518 0.2079 0.2491 1.0000
Table 71

In Python
### Correlation between variables Q1 to Q10
Code

survey_data_corr = survey_data.corr(method='spearman')
survey_data_corr

Q1 Q2 Q3 Q4 Q5 Q6 Q7
Q1 1.000000 0.221737 0.223101 0.256695 0.432009 0.404096 0.254117
Q2 0.221737 1.000000 0.312831 0.265311 0.239049 0.193397 0.138398
Q3 0.223101 0.312831 1.000000 0.154184 0.213342 0.259665 0.235240
Q4 0.256695 0.265311 0.154184 1.000000 0.423595 0.145878 0.081694
Q5 0.432009 0.239049 0.213342 0.423595 1.000000 0.382424 0.157458
Q6 0.404096 0.193397 0.259665 0.145878 0.382424 1.000000 0.228059
Q7 0.254117 0.138398 0.235240 0.081694 0.157458 0.228059 1.000000
Q8 0.271804 0.225324 0.237712 0.316570 0.438128 0.221814 0.195582
Q9 0.537406 0.027119 0.072103 0.148947 0.441893 0.308125 0.325718
Output

Q10 0.284481 0.150652 0.248648 0.251149 0.267063 0.338883 0.251794

Q8 Q9 Q10
Q1 0.271804 0.537406 0.284481
Q2 0.225324 0.027119 0.150652
Q3 0.237712 0.072103 0.248648
Q4 0.316570 0.148947 0.251149
Q5 0.438128 0.441893 0.267063
Q6 0.221814 0.308125 0.338883
Q7 0.195582 0.325718 0.251794
Q8 1.000000 0.275094 0.207895
Q9 0.275094 1.000000 0.249120
Q10 0.207895 0.249120 1.000000
Table 72

The outputs above show the matrix of correlations for Q1 to Q10 variables. The correlation values
(between different questions) varies from 0.027 (Q2 and Q9) to 0.537 (Q1 and Q9). The data of this
matrix and this correlation values suggest the variables could be reduced down to at least two underlying
variables or factors. This is a preliminary conclusion, as the correlation values vary from very little
correlation to moderate correlation between the pairs of survey questions. Nonetheless, we can have more
factors, and additional research should be addressed with this data.
In the following subsections, we present some suggestions to do factor analysis, in R and Python.

Sampling Adequacy
Before starting factor analysis, it should be checked whether it is appropriate to the data in the study. For
this verification, two methods could be applied: the Bartlett sphericity test and the KMO Measure.

Bartlett Sphericity test

Exploratory factor analysis is only useful if the matrix of population correlation is statistically different
from the identity matrix. If these are equal, the variables are few interrelated, i.e., the specific factors
explain the greater proportion of the variance and the common factors are unimportant. Therefore, it
should be defined when the correlations between the original variables are sufficiently high. Thus, the
factor analysis is useful in estimation of common factors. With this in mind, the Bartlett Sphericity test
can be used. The hypotheses are:
 120

H! : the matrix of population correlations is equal to the identity matrix

H! : the matrix of population correlations is different from the identity matrix.

This test can be calculated in R or Python. However, with some packages, it can be made more quickly. It
is the case of the PSYCH R’s package, which proposes Bartlett's test. In Python, there is a need to
program the Bartlett Sphericity test function. To our best knowledge, there is no implementation of this
test in any free module. Thus, Bartlett's test statistic formula is:

2𝑝 + 5
𝑋! = − 𝑛 − 1 − ×𝑙𝑛 𝑅
6
(!!!)
where 𝑅 is the determinant of the correlation matrix, and   𝑝  ×     is the number of degrees of
!
freedom with 𝑝 as the number of variables. The programming code for this calculation is:

In R
### Bartlett Sphericity test
Code

library (psych)
[Link](correlation, n=nrow(data))
$chisq
[1] 410.2728
Output

$[Link]
[1] 1.949995e-60

$df
[1] 45
Table 73

In Python
### Bartlett Sphericity test

import numpy as np
import math as math
import [Link] as stats

# Generate Identity Matrix

# 10x10 Identity Matrix
indentity = [Link](10)
Code

# The Bartlett test

n = survey_data.shape[0]
p = survey_data.shape[1]
chi2 = -(n-1-(2*p+5)/6)*[Link]([Link](survey_data_corr))
ddl = p*(p-1)/2
pvalue = [Link](chi2 , ddl)
chi2
ddl
pvalue
chi2
Out[12]: 410.27280642443156
Output

ddl
Out[13]: 45.0

pvalue
Out[14]: 8.7335941050291506e-61
Table 74
 121

With the previous results, it is possible to conclude the rejection of the null hypothesis (p<0.05), and
therefore, the matrix of population correlations is different from the identity matrix. This difference
suggests that factorial analysis is appropriate to our data.
However, Bartlett Sphericity test is rarely used since it is very sensitive to the dimension of the sample. In
large samples, 𝐻! is frequently rejected even when the correlations are very small. The test also requires
the multivariate normal distribution of the variables, and it is very sensitive to this assumption’s violation.

KMO Measure
Given the limitations of Bartlett Sphericity test, there are other methods with the same goal that can be
used to assess the quality of data. A widely used method is the "measure of the adequacy of sampling
Kaiser-Meyer-Olkin" (KMO). KMO checks if it is possible to factorize the primary variables efficiently.
But it is based on another idea.
The correlation matrix is always the starting point. The variables are more or less correlated, but the
others can influence the correlation between the two variables. Hence, with KMO, the partial correlation
is used to measure the relation between two variables by removing the effect of the remaining variables.
The partial correlation matrix can be obtained from the correlation matrix. Considering the inverse of the
correlation matrix as 𝑅 !!   =   (𝑣!" ), the partial correlation as 𝐴   =   (𝑎!" ), and the observed correlation
matrix as 𝑅 = (𝑟!" ), we have:

𝑣!"
𝑎!" = −
𝑣!! ×𝑣!!
Thus, the overall KMO index is computed as:
!
! !!! 𝑟!"
𝐾𝑀𝑂 = ! !
! !!! 𝑟!" + ! !!! 𝑎!"

and the KMO index per variable to detect those which are not related to the others is:
!
!!! 𝑟!"
𝐾𝑀𝑂! = ! !
!!! 𝑟!" + !!! 𝑎!"

KMO returns values between 0 and 1. A rule of thumb for interpreting the statistic:
• KMO values between 0.8 and 1 indicate the sampling is adequate.
• KMO values less than 0.6 indicate the sampling is not appropriate and that remedial action should
be taken. Some authors put this value at 0.5, so the researcher should use his judgment for values
between 0.5 and 0.6.
• KMO Values close to zero means that there are high partial correlations compared to the sum of
correlations. In other words, there are widespread correlations, which are a large problem for
factor analysis.

For reference, Kaiser suggested the following classification of the results:

• 0 to 0.49 unacceptable
• 0.50 to 0.59 miserable
• 0.60 to 0.69 mediocre
• 0.70 to 0.79 middling
• 0.80 to 0.89 meritorious.
 122

• 0.90 to 1.00 marvelous.

Similar to Bartlett Sphericity test, the KMO measure could be calculated in R or Python. However, with
the package used above (PSYCH), in R, the code is much simpler. In Python, this test will be
programmed since, to our best knowledge, a module with KMO was not available.

In R
### KMO Measure
Code

library (psych)
KMO(correlation)
Kaiser-Meyer-Olkin factor adequacy
Output

Call: KMO(r = correlation)

Overall MSA = 0.8
MSA for each item =
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10
0.81 0.77 0.79 0.77 0.80 0.84 0.79 0.86 0.71 0.86
Table 75

In Python
 123

### KMO Measure

import numpy as np
import math as math

### a) Global KMO

# Inverse of the correlation matrix
# dataset_corr is the correlation matrix of the survey results
corr_inv = [Link](dataset_corr)
# number of rows and number of columns
nrow_inv_corr, ncol_inv_corr = dataset_corr.shape

# Partial correlation matrix

A = [Link]((nrow_inv_corr,ncol_inv_corr))
for i in range(0,nrow_inv_corr,1):
for j in range(i,ncol_inv_corr,1):
#above the diagonal
A[i,j] = - (corr_inv[i,j])/([Link](corr_inv[i,i] * corr_inv[j,j]))
#below the diagonal
A[j,i] = A[i,j]
Code

#transform to an array of arrays ("matrix" with Python)

dataset_corr = [Link](dataset_corr)

#KMO value
kmo_num = [Link]([Link](dataset_corr))-
[Link]([Link]([Link](dataset_corr)))
kmo_denom = kmo_num + [Link]([Link](A)) - [Link]([Link]([Link](A)))
kmo_value = kmo_num / kmo_denom
print(kmo_value)

### b) KMO per variable

#creation of an empty vector to store the results per variable. The size of the vector is equal to the number
#...of variables
kmo_j = [None]*dataset_corr.shape[1]

for j in range(0, dataset_corr.shape[1]):

kmo_j_num = [Link](dataset_corr[:,[j]] ** 2) - dataset_corr[j,j] ** 2
kmo_j_denom = kmo_j_num + [Link](A[:,[j]] ** 2) - A[j,j] ** 2
kmo_j[j] = kmo_j_num / kmo_j_denom
print(kmo_j)

a) Global KMO
...: print(kmo_value)
Output

0.798844102413

b) KMO per item

...: print(kmo_j)
0.812160468405 0.774161264483 0.786819432663 0.766251123086 0.800579196084
0.842927745203 0.792010173432 0.862037322891 0.714795031915 0.856497242574
Table 76

With KMO = 0.8, the degree of common variance in our dataset is “meritorious”. All variables have
KMO higher than 0.5, and therefore, the factor analysis is appropriate to this data. However, if KMO is
less than 0.5, a detailed discussion of the variables should be made.

Retained factors
Since it is possible to make a factor analysis to the data in the study, the next step is to find the weights
for a set of latent factors. However, this type of mathematical model has multiple possible solutions. This
 124

problem is referred to as indeterminacy of Exploratory Factor Analysis (EFA) equation caused by the
problem of factors rotation. Therefore, whenever a solution is not interpretable, it can be made a rotation
of factors (multiplication by an orthogonal matrix). The “rotation” is equivalent to the translation of
factorial axes in the factorial space without changing the orientation of the vectors representing the
variables.

Number of factors to be retained

Before starting to analyze factors, it is important to know how many factors should be maintained.
Several studies were developed to decide this number. Hayton et al. (2004) state three reasons why this
decision is so important. Firstly, it can affect EFA results more than other decisions, such as selecting an
extraction method or the factor rotation method, since there is evidence of the relative robustness of EFA
with regards to these matters. Secondly, the EFA requires that balance is struck between “reducing” and
adequately “representing” the correlations that exist in a group of variables. Therefore, its very usefulness
depends on distinguishing relevant factors from trivial ones. Lastly, an error regarding selecting the
number of factors can significantly alter the solution and the interpretation of EFA results. The extraction
of fewer factors can lead to the loss of relevant information and a substantial distortion in the solution (for
example, in the loading variables). On the other hand, although less problematic, the extraction of an
excessive number of factors can lead to factors with a substantial less loading. Thus, it can be difficult to
interpret and/or replicate.
Given the importance of this decision, different methods have been proposed to determine the number of
factors to retain.

Kaiser criterion: method suggested by Kaiser (1960). According to his rule, only factors with
eigenvalues greater than one are retained for interpretation. Despite the simplicity of this approach, many
authors agree that it is problematic and inefficient when it comes to determining the number of factors
(Ledesma and Valero-Mora, 2007). For example, it doesn’t make much sense to regard a factor with an
eigenvalue of 1.01 as “major” and one with an eigenvalue of .99 as “trivial”. This method should be used
together with other methods.

In R
### Kaiser criterion
Code

library (psych)
eigen(correlation)
 125

$values
[1] 3.3669166 1.2136498 1.0776487 0.8213321 0.7951511 0.7124396 0.6119054 0.5670223
[9] 0.4679005 0.3660340

$vectors
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] -0.3853256 0.256021260 0.04702543 0.05434129 -0.34124508 -0.24403301
[2,] -0.2385566 -0.554897142 -0.12377747 -0.10793243 -0.37755610 -0.46948278
[3,] -0.2590495 -0.384579932 -0.47012641 -0.05655833 -0.10685503 0.37008699
[4,] -0.2827407 -0.338060670 0.45012288 0.08480756 0.36816709 -0.33303133
[5,] -0.3970827 0.003995701 0.37177858 0.02653913 -0.09457523 0.14696031
[6,] -0.3376002 0.129020624 -0.20801818 0.46735905 -0.29588083 0.23702543
Output

[7,] -0.2552794 0.207102791 -0.48687588 -0.53430300 0.33095236 -0.25730614

[8,] -0.3229872 -0.162204890 0.26236628 -0.41518560 0.15971844 0.56139937
[9,] -0.3385482 0.527492060 0.11965551 -0.17450361 -0.09880217 -0.10303279
[10,] -0.3030087 0.015600699 -0.23905186 0.51726913 0.59392502 -0.04888472
[,7] [,8] [,9] [,10]
[1,] -0.31897190 0.1916059 0.514234180 -0.44935505
[2,] 0.32625111 0.3132422 -0.159056263 0.12404674
[3,] -0.61538208 -0.1078221 -0.082569260 0.12350375
[4,] -0.19358060 -0.4319708 0.251440196 0.24943628
[5,] 0.01215364 -0.1942442 -0.650994313 -0.45953434
[6,] 0.46031902 -0.4160287 0.190036142 0.20705075
[7,] 0.20849609 -0.3364610 -0.009101036 -0.19213969
[8,] 0.29342881 0.3084268 0.324548292 0.03025459
[9,] -0.18018807 0.1977266 -0.250906346 0.63833622
[10,] 0.03960368 0.4568397 -0.109310158 -0.07663455
Table 77

In Python
### Kaiser criterion
Code

[Link](survey_data_corr)

[Link](survey_data_corr)
Out[20]:
(array([ 3.36691657, 1.21364977, 0.366034 , 1.0776487 , 0.46790045,
0.56702233, 0.61190541, 0.71243962, 0.79515105, 0.8213321 ]),
matrix([[ 0.38532555, 0.25602126, 0.44935505, -0.04702543, -0.51423418,
0.19160593, -0.3189719 , 0.24403301, 0.34124508, 0.05434129],
[ 0.2385566 , -0.55489714, -0.12404674, 0.12377747, 0.15905626,
0.31324219, 0.32625111, 0.46948278, 0.3775561 , -0.10793243],
[ 0.25904952, -0.38457993, -0.12350375, 0.47012641, 0.08256926,
-0.10782206, -0.61538208, -0.37008699, 0.10685503, -0.05655833],
[ 0.28274072, -0.33806067, -0.24943628, -0.45012288, -0.2514402 ,
Output

-0.43197081, -0.1935806 , 0.33303133, -0.36816709, 0.08480756],

[ 0.39708267, 0.0039957 , 0.45953434, -0.37177858, 0.65099431,
-0.19424424, 0.01215364, -0.14696031, 0.09457523, 0.02653913],
[ 0.33760022, 0.12902062, -0.20705075, 0.20801818, -0.19003614,
-0.41602867, 0.46031902, -0.23702543, 0.29588083, 0.46735905],
[ 0.25527937, 0.20710279, 0.19213969, 0.48687588, 0.00910104,
-0.33646105, 0.20849609, 0.25730614, -0.33095236, -0.534303 ],
[ 0.32298719, -0.16220489, -0.03025459, -0.26236628, -0.32454829,
0.30842676, 0.29342881, -0.56139937, -0.15971844, -0.4151856 ],
[ 0.33854818, 0.52749206, -0.63833622, -0.11965551, 0.25090635,
0.1977266 , -0.18018807, 0.10303279, 0.09880217, -0.17450361],
[ 0.30300873, 0.0156007 , 0.07663455, 0.23905186, 0.10931016,
0.45683972, 0.03960368, 0.04888472, -0.59392502, 0.51726913]]))
Table 78

The outputs are divided into “values” and “vectors”, corresponding to eigenvalues and eigenvectors,
respectively. As it is possible to observe, there are three eigenvalues higher than 1. Please note that, in
Python, eigenvalues are not ordered. This suggests retaining three factors.

Scree plot: method proposed by Cattell (1966), which involves the visual exploration of a graphical
representation of the eigenvalues. In this approach, the eigenvalues are presented in descending order and
linked with a line. Afterward, the graph is examined to determine the point at which the last significant
drop or break takes place - in other words, where the line levels off. The logic behind this method is that
the point divides the critical or major factors from the minor or unimportant factors. Scree plot has been
 126

criticized for its subjectivity since there is not an objective definition of the cutoff point between the
important and trivial factors. Indeed, some cases may present several drops and possible cutoff points,
such that the graph may be ambiguous and difficult to interpret.

In R
### Scree plot criterion
Code

library(nFactors)
scree(correlation, hline=-1) # hline=-1 draw a horizontal line at -1

Scree plot
Eigen values of factors and components

PC
FA
3.0
2.0
Output

1.0
0.0

2 4 6 8 10
factor or component number

Figure 62 Scree plot in R

Table 79

In Python
### Scree plot criterion (calling R functions from Python)

# See [Link]
# See [Link] to install rpy2
import rpy2 as rpy2
from [Link] import importr
import [Link] as ro
import [Link] as com

# Changing R's directory to where is the data

ro.r('setwd("C:/Users/Rui Sarmento/Documents/Livro Cybertech/Dados e Code")')
Code

# Reading the data with R

ro.r('data_df <- [Link]("[Link]",sep=";")')

# Retrieving the correlation matrix of the survey answers

ro.r('correlation <- cor(data_df[,paste("Q",1:10,sep="")], method="spearman")')

### Scree plot criterion

psych = importr('psych')

# Scree function call with R

ro.r('scree(correlation, hline=-1)') # hline=-1 draw a horizontal line at -1')
library(nFactors)
scree(correlation, hline=-1) # hline=-1 draw a horizontal line at -1
Output 127

Figure 63 Scree plot in Python

Table 80

The observation of the previous scree plots suggests that at least two factors should be retained. In this
point, the curve makes an “elbow” toward less steep decline. Other suggestion is to consider four factors
(little elbow).

Variance explained criteria: a method based on similar conceptual structure is to retain the number of
factors that account for a certain percent of extracted variance. The literature varies on how much
variance should be explained before the number of factors is sufficient. The majority suggests that 75-
90% of the variance should be accounted. However, some statisticians indicate as much as 50% of the
variance explained is acceptable. As with any criteria method solely depending on variance, this
seemingly full standard must be viewed about to the foundational differences between extraction
methods.

In R
### Explained variance for each component
Code

pc <- prcomp(survey,scale.=F)
summary(pc)
Importance of components:
PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8
Standard deviation 2.0616 1.1715 1.0936 0.97953 0.90417 0.89021 0.82018 0.78209
Output

Proportion of Variance 0.3661 0.1182 0.1030 0.08264 0.07041 0.06825 0.05794 0.05268
Cumulative Proportion 0.3661 0.4843 0.5873 0.66990 0.74031 0.80856 0.86650 0.91918
PC9 PC10
Standard deviation 0.69473 0.67509
Proportion of Variance 0.04157 0.03925
Cumulative Proportion 0.96075 1.00000
Table 81
 128

In Python
### Explained Variance for each component

# See [Link]
Code

from [Link] import PCA

import pandas as pd
pca = PCA().fit(survey_data)
pca.explained_variance_ratio_
Output

Out[22]:
array([ 0.36606121, 0.11819385, 0.10300932, 0.08263505, 0.07041005,
0.06825262, 0.05793681, 0.05268024, 0.04156898, 0.03925188])

Table 82

The previous outputs give the importance of each component for the number of variables in the study, i.e.,
10. To have 75% of the explained variance, it would be necessary to consider six components. However,
for example, in psychology or sociology, these values are difficult to reach with a small number of
components. In this case, the minimum acceptable value (50%) should be considered. Thus, in our case
study, three factors should be retained.
To analyze three methods discussed above, in the following analysis, three factors will be considered.
However, the reader must have critical thought and check if the number of suggested factors makes sense
in the scope of the problem that is being analyzed.

To estimate the matrix of factor weights, it is necessary to have an estimate of the communalities. Among
the various methods for this estimation, the most popular are Principal Component Analysis, Principal
Axis, and Maximum Likelihood Factor Analysis.

Principal Component Method

The principal component method is based on the determination of the eigenvalues and eigenvectors of the
correlation matrix. First, an initial estimate is provided, which is the maximum value of the correlation
(i.e., 1). Subsequently, the number of principal components to retain is determined.

Principal Axis Method of Factor Extraction

It is an iterative PCA application to the matrix where communalities stand on the diagonal in place of 1’s.
Each iteration refines communalities further until they converge. In doing so, the method seeks to explain
variance, not pairwise correlations. Principal Axis method has the advantage in that it can, like PCA,
analyze not only correlations but also covariance.

Maximum Likelihood Method (ML)

Assumes that correlation came from a population having multivariate normal distribution (other methods
make no such assumption) and hence the residuals of correlation coefficients must be normally distributed
around 0. The loadings are iteratively estimated by ML Communalities approach under the above
assumption. The treatment of correlations is weighted by uniqueness values. While other methods just
analyze the sample as it is, ML method allows some inference about the population, some fit indices, and
confidence intervals are usually computed along with it.

Succinctly, in most cases, both Principal Component method and Principal Axis method, lead to the same
factor structure, and the difference between the methods is mostly conceptual. The Principal Component
is the most commonly used method. However, the Principal Axis method is conceptually more attractive,
since it assumes a factor structure composed of common factors and specific factors. Consequently, with
 129

this method, it is possible to obtain higher factor weights, which facilitates the interpretation of factors.
This occurs because they do not have to include the specificity of each variable during the extraction of
factors. However, this method is more affected by the indeterminacy of the factors and may cause
obtaining very different factor structures from the original data. This disadvantage of the method is
particularly penalizing in EFA since it is heavily dependent on the performed sampling. Finally, the
maximum likelihood method requires that variables under study present multivariate normal distribution,
which is not always easy to validate. This is a reason why the Principal Component method is somewhat
recommended. However, it has the advantage that, for large samples, allows the calculation of indices to
evaluate the quality of the factor model.
Therefore, the most common method is used in this book: Principal Component method.

In R
### Principal Component method
Code

library (psych)
principal(correlation,nfactors=3, rotate="none")
Principal Components Analysis
Call: principal(r = correlation, nfactors = 3, rotate = "none")
Standardized loadings (pattern matrix) based upon correlation matrix
PC1 PC2 PC3 h2 u2 com
Q1 0.71 -0.28 -0.05 0.58 0.42 1.3
Q2 0.44 0.61 0.13 0.58 0.42 1.9
Q3 0.48 0.42 0.49 0.64 0.36 3.0
Q4 0.52 0.37 -0.47 0.63 0.37 2.8
Q5 0.73 -0.01 -0.39 0.68 0.32 1.5
Q6 0.62 -0.14 0.22 0.45 0.55 1.4
Q7 0.47 -0.23 0.50 0.53 0.47 2.4
Q8 0.59 0.18 -0.27 0.46 0.54 1.6
Output

Q9 0.62 -0.58 -0.12 0.74 0.26 2.1

Q10 0.56 -0.02 0.25 0.37 0.63 1.4

PC1 PC2 PC3

SS loadings 3.37 1.21 1.08
Proportion Var 0.34 0.12 0.11
Cumulative Var 0.34 0.46 0.57
Proportion Explained 0.60 0.21 0.19
Cumulative Proportion 0.60 0.81 1.00

Mean item complexity = 1.9

Test of the hypothesis that 3 components are sufficient.

The root mean square of the residuals (RMSR) is 0.1

Fit based upon off diagonal values = 0.87

Table 83

In Python
 130

### Principal Component method (using R)

import rpy2 as rpy2

from [Link] import importr
import [Link] as ro
import [Link] as com

# Changing R's directory to where data is

ro.r('setwd("C:/Users/Rui Sarmento/Documents/Livro Cybertech/Dados e Code")')

# Reading data with R

Code

ro.r('data_df <- [Link]("[Link]",sep=";")')

# Retrieving the correlation matrix of the survey answers

ro.r('correlation <- cor(data_df[,paste("Q",1:10,sep="")], method="spearman")')

# Uses of psych R's package

ro.r('library (psych)')

# Calling function principal with R

print(ro.r('principal(correlation,nfactors=3, rotate="none")'))
library (psych)
principal(correlation,nfactors=3, rotate="none")
Principal Components Analysis
Call: principal(r = correlation, nfactors = 3, rotate = "none")
Standardized loadings (pattern matrix) based upon correlation matrix
PC1 PC2 PC3 h2 u2 com
Q1 0.71 -0.28 -0.05 0.58 0.42 1.3
Q2 0.44 0.61 0.13 0.58 0.42 1.9
Q3 0.48 0.42 0.49 0.64 0.36 3.0
Q4 0.52 0.37 -0.47 0.63 0.37 2.8
Q5 0.73 -0.01 -0.39 0.68 0.32 1.5
Q6 0.62 -0.14 0.22 0.45 0.55 1.4
Q7 0.47 -0.23 0.50 0.53 0.47 2.4
Q8 0.59 0.18 -0.27 0.46 0.54 1.6
Output

Q9 0.62 -0.58 -0.12 0.74 0.26 2.1

Q10 0.56 -0.02 0.25 0.37 0.63 1.4

PC1 PC2 PC3

SS loadings 3.37 1.21 1.08
Proportion Var 0.34 0.12 0.11
Cumulative Var 0.34 0.46 0.57
Proportion Explained 0.60 0.21 0.19
Cumulative Proportion 0.60 0.81 1.00

Mean item complexity = 1.9

Test of the hypothesis that 3 components are sufficient.

The root mean square of the residuals (RMSR) is 0.1

Fit based upon off diagonal values = 0.87

Table 84

Looking at standardized loadings (pattern matrix) based upon correlation matrix, in the previous output,
the first three columns indicate the variable's weight in each defined component. The weights allow
defining the variables belonging to each component. Furthermore, h2 and u2 are the values of
communalities and uniquenesses, respectively. Communality is the proportion of each variable's variance
that can be explained by the factors. Uniqueness gives the percentage of the common variance of the
variable not associated with the factors.

𝑈𝑛𝑖𝑞𝑢𝑒𝑛𝑒𝑠𝑠   =  1   −  𝑐𝑜𝑚𝑚𝑢𝑛𝑎𝑙𝑖𝑡𝑦

Analyzing the communality values, it is possible to verify that all values are higher than 0.5, except Q6,
Q8, and Q10. This means that except these three indicated variables, the percentage of the variance of
 131

each variable explained by common factors is greater than 50%. Thus, with some statistical rigor, these
three variables should be eliminated. However, it is up to the reader to decide on this rigor, that is, to
determine what is an acceptable lower limit. We should point out that this value should not be less than
30%. Analyzing the weights of each variable in the factors, the reader should check the factor with greater
weight for the variable. The variable should belong to this factor. Nevertheless, in the case of Q3, the
weights in PC1, PC2, and PC3 are 0.48, 0.42, and 0.49, respectively. This may lead to some doubts
regarding the model because there is no sense to decide where the Q3 variable should be included. To
eliminate these doubts, the results should be analyzed after a factor rotation (next subsection).
Check the same results considering the elimination of the variables whose communality values are less
than 0.5. The results remain not interpretable.

In R
### Principal Component method

[Link]<-correlation[!(colnames(correlation) %in% c("Q6", "Q8",

Code

"Q10")),!(rownames(correlation) %in% c("Q6", "Q8", "Q10"))]

[Link]

library (psych)
principal([Link],nfactors=3, rotate="none")
Principal Components Analysis
Call: principal(r = [Link], nfactors = 3, rotate = "none")
Standardized loadings (pattern matrix) based upon correlation matrix
PC1 PC2 PC3 h2 u2 com
Q1 0.75 -0.25 -0.02 0.63 0.37 1.2
Q2 0.47 0.65 0.06 0.64 0.36 1.8
Q3 0.47 0.50 0.45 0.67 0.33 3.0
Q4 0.55 0.30 -0.53 0.67 0.33 2.5
Q5 0.74 -0.05 -0.36 0.68 0.32 1.4
Q7 0.48 -0.17 0.65 0.68 0.32 2.0
Q9 0.66 -0.59 0.02 0.79 0.21 2.0
Output

PC1 PC2 PC3

SS loadings 2.53 1.20 1.03
Proportion Var 0.36 0.17 0.15
Cumulative Var 0.36 0.53 0.68
Proportion Explained 0.53 0.25 0.22
Cumulative Proportion 0.53 0.78 1.00

Mean item complexity = 2

Test of the hypothesis that 3 components are sufficient.

The root mean square of the residuals (RMSR) is 0.12

Fit based upon off diagonal values = 0.82

Table 85

In Python
### Principal Component method

# New correlation matrix

ro.r('[Link]<-correlation[!(colnames(correlation) %in% c("Q6", "Q8",
Code

"Q10")),!(rownames(correlation) %in% c("Q6", "Q8", "Q10"))]')

ro.r('library(psych)')

# Calling function principal with R

print(ro.r('principal([Link],nfactors=3, rotate="none")'))
 132

Principal Components Analysis

Call: principal(r = [Link], nfactors = 3, rotate = "none")
Standardized loadings (pattern matrix) based upon correlation matrix
PC1 PC2 PC3 h2 u2 com
Q1 0.75 -0.25 -0.02 0.63 0.37 1.2
Q2 0.47 0.65 0.06 0.64 0.36 1.8
Q3 0.47 0.50 0.45 0.67 0.33 3.0
Q4 0.55 0.30 -0.53 0.67 0.33 2.5
Q5 0.74 -0.05 -0.36 0.68 0.32 1.4
Q7 0.48 -0.17 0.65 0.68 0.32 2.0
Q9 0.66 -0.59 0.02 0.79 0.21 2.0
Output

PC1 PC2 PC3

SS loadings 2.53 1.20 1.03
Proportion Var 0.36 0.17 0.15
Cumulative Var 0.36 0.53 0.68
Proportion Explained 0.53 0.25 0.22
Cumulative Proportion 0.53 0.78 1.00

Mean item complexity = 2

Test of the hypothesis that 3 components are sufficient.

The root mean square of the residuals (RMSR) is 0.12

Fit based upon off diagonal values = 0.82

Table 86

Factor Rotations
EFA solution is not always interpretable. The factor weights of the variables in common factors can be
such that it is not possible to assign a meaning to extracted empirical factors. From the mathematical point
of view, the extracted factors are not the only existing ones, and an orthogonal matrix can be multiplied
by the matrix of factor weights. The multiplication corresponds to the rotation of the factorial axes and
does not alter the communalities or the specific variance, i.e., does not modify the data structure.
The factorial axes are mathematical structures and not laws of nature. Hence, there is no reason for an
axis system to be preferred over another axis system. Moreover, the best axis system is one that produces
a factor solution easily interpretable. There are several methods to make the rotation of the factorial axes,
including the Varimax method, the Quartimax method and the Oblimin method.

Varimax
Varimax, which was developed by Kaiser (1958), is indubitably the most popular rotation method by far.
For Varimax, a simple solution means that each factor has a small number of large loadings and a large
number of zero (or low) loadings. This simplifies the interpretation because, after a varimax rotation, each
original variable tends to be associated with one (or a small number) of factors, and each factor represents
only a limited number of variables. Additionally, the factors can often be interpreted from the opposition
of few variables with positive loadings to few variables with negative loadings (Abdi, 2003).

Quartimax
Quartimax rotation is a form of orthogonal rotation used to transform vectors associated with principal
component analysis or factor analysis to a simple structure. It is a particular case of orthomax rotation,
which maximizes the sums of squares of the coefficients across the resultant vectors for each of the
original variables, Quartimax is opposed to varimax, which maximizes the sums of squares of the
coefficients within each of the resultant vectors.

Oblimin
Oblimin rotation is a general form for obtaining oblique rotations used to transform vectors associated
with principal component analysis or factor analysis to a simple structure. Oblimin is similar to the
Orthomax rotation procedures used in orthogonal rotation in that, it too, includes an arbitrary constant
 133

used to obtain different rotational properties. While most orthogonal rotations use some form of
Orthomax rotation, this is no longer the case with Oblimin rotation for the oblique case.

For the next step, the varimax method was used. The results are presented below.

In R
### Principal Component method with varimax rotation
Code

library (psych)
principal(correlation,nfactors=3, rotate="varimax")
Principal Components Analysis
Call: principal(r = correlation, nfactors = 3, rotate = "varimax")
Standardized loadings (pattern matrix) based upon correlation matrix
PC3 PC1 PC2 h2 u2 com
Q1 0.66 0.39 0.01 0.58 0.42 1.6
Q2 -0.01 0.35 0.68 0.58 0.42 1.5
Q3 0.25 0.05 0.76 0.64 0.36 1.2
Q4 -0.02 0.77 0.18 0.63 0.37 1.1
Q5 0.38 0.73 0.02 0.68 0.32 1.5
Q6 0.60 0.18 0.23 0.45 0.55 1.5
Q7 0.65 -0.15 0.28 0.53 0.47 1.5
Q8 0.22 0.61 0.18 0.46 0.54 1.4
Output

Q9 0.75 0.30 -0.29 0.74 0.26 1.6

Q10 0.49 0.15 0.32 0.37 0.63 1.9

PC3 PC1 PC2

SS loadings 2.29 1.95 1.42
Proportion Var 0.23 0.19 0.14
Cumulative Var 0.23 0.42 0.57
Proportion Explained 0.40 0.34 0.25
Cumulative Proportion 0.40 0.75 1.00

Mean item complexity = 1.5

Test of the hypothesis that 3 components are sufficient.

The root mean square of the residuals (RMSR) is 0.1

Fit based upon off diagonal values = 0.87

Table 87

In Python
### Principal Component method with varimax rotation (with R)

import rpy2 as rpy2

from [Link] import importr
import [Link] as ro
import [Link] as com

# Changing R's directory to where is the data

Code

ro.r('setwd("C:/Users/Rui Sarmento/Documents/Livro Cybertech/Dados e Code")')

# Reading the data with R

ro.r('data_df <- [Link]("[Link]",sep=";")')

# Retrieving the correlation matrix of the survey answers

ro.r('correlation <- cor(data_df[,paste("Q",1:10,sep="")], method="spearman")')

# Calling function principal with varimax rotation (with R)

print(ro.r('principal(correlation,nfactors=3, rotate="varimax")'))
 134

Principal Components Analysis

Call: principal(r = correlation, nfactors = 3, rotate = "varimax")
Standardized loadings (pattern matrix) based upon correlation matrix
PC3 PC1 PC2 h2 u2 com
Q1 0.66 0.39 0.01 0.58 0.42 1.6
Q2 -0.01 0.35 0.68 0.58 0.42 1.5
Q3 0.25 0.05 0.76 0.64 0.36 1.2
Q4 -0.02 0.77 0.18 0.63 0.37 1.1
Q5 0.38 0.73 0.02 0.68 0.32 1.5
Q6 0.60 0.18 0.23 0.45 0.55 1.5
Q7 0.65 -0.15 0.28 0.53 0.47 1.5
Output

Q8 0.22 0.61 0.18 0.46 0.54 1.4

Q9 0.75 0.30 -0.29 0.74 0.26 1.6
Q10 0.49 0.15 0.32 0.37 0.63 1.9

PC3 PC1 PC2

SS loadings 2.29 1.95 1.42
Proportion Var 0.23 0.19 0.14
Cumulative Var 0.23 0.42 0.57
Proportion Explained 0.40 0.34 0.25
Cumulative Proportion 0.40 0.75 1.00

Mean item complexity = 1.5

Test of the hypothesis that 3 components are sufficient.
The root mean square of the residuals (RMSR) is 0.1
Fit based upon off diagonal values = 0.87
Table 88

In this case, PC3 is composed of Q1, Q6, Q7, Q9, and Q10 variables. PC1 is composed of Q4, Q5, and
Q8 variables. PC2 is composed of Q2 and Q3 variables.
The RMSR value equal to 0.1 means that the retained factors are appropriate to describe the correlation
structure. As reference values, the model is considered unacceptable when RMSR value is greater than
0.1. It is considered as excellent when RMSR value is less than 0.05.
Check the same results considering the elimination of the variables whose communality values are less
than 0.5.

In R
### Principal Component method
Code

library (psych)
principal([Link],nfactors=3, rotate="varimax")
Principal Components Analysis
Call: principal(r = [Link], nfactors = 3, rotate = "varimax")
Standardized loadings (pattern matrix) based upon correlation matrix
PC3 PC1 PC2 h2 u2 com
Q1 0.70 0.33 0.15 0.63 0.37 1.5
Q2 -0.07 0.40 0.69 0.64 0.36 1.6
Q3 0.12 0.05 0.81 0.67 0.33 1.1
Q4 0.08 0.80 0.16 0.67 0.33 1.1
Q5 0.48 0.66 0.09 0.68 0.32 1.9
Q7 0.62 -0.31 0.46 0.68 0.32 2.4
Output

Q9 0.87 0.16 -0.11 0.79 0.21 1.1

PC3 PC1 PC2

SS loadings 1.88 1.47 1.40
Proportion Var 0.27 0.21 0.20
Cumulative Var 0.27 0.48 0.68
Proportion Explained 0.40 0.31 0.30
Cumulative Proportion 0.40 0.70 1.00

Mean item complexity = 1.5

Test of the hypothesis that 3 components are sufficient.
The root mean square of the residuals (RMSR) is 0.12
Fit based upon off diagonal values = 0.82
Table 89
 135

In Python
### Principal Component method
Code

# New correlation matrix

ro.r('[Link]<-correlation[!(colnames(correlation) %in% c("Q6", "Q8",
"Q10")),!(rownames(correlation) %in% c("Q6", "Q8", "Q10"))]')
print(ro.r('principal([Link],nfactors=3, rotate="varimax")'))
Principal Components Analysis
Call: principal(r = [Link], nfactors = 3, rotate = "varimax")
Standardized loadings (pattern matrix) based upon correlation matrix
PC3 PC1 PC2 h2 u2 com
Q1 0.70 0.33 0.15 0.63 0.37 1.5
Q2 -0.07 0.40 0.69 0.64 0.36 1.6
Q3 0.12 0.05 0.81 0.67 0.33 1.1
Q4 0.08 0.80 0.16 0.67 0.33 1.1
Q5 0.48 0.66 0.09 0.68 0.32 1.9
Q7 0.62 -0.31 0.46 0.68 0.32 2.4
Output

Q9 0.87 0.16 -0.11 0.79 0.21 1.1

PC3 PC1 PC2
SS loadings 1.88 1.47 1.40
Proportion Var 0.27 0.21 0.20
Cumulative Var 0.27 0.48 0.68
Proportion Explained 0.40 0.31 0.30
Cumulative Proportion 0.40 0.70 1.00

Mean item complexity = 1.5

Test of the hypothesis that 3 components are sufficient.
The root mean square of the residuals (RMSR) is 0.12
Fit based upon off diagonal values = 0.82
Table 90

The previous outputs show that communality values are always greater than 60%. Moreover, PC3 is
composed of Q1, Q7, and Q9 variables. PC1 consists of Q4 and Q5 variables. PC2 is composed of Q2 and
Q3 variables. In this case, RMSR is equal to 0.12, and then the model is unacceptable.

Remembering the “Dataset” chapter, we can clearly relate each factor to a subject of study in the
questionnaire. Therefore, the first factor groups the questions related to research tools, the second factor
groups the questions related to research methods. Finally, the third factor groups the productivity related
questions. Consequently, in the study of this book, all variables must be considered, and the factors are:

Factor 1: Q1, Q6, Q7, Q9, Q10 ! Research Tools

Factor 2: Q4, Q5, Q8 ! Research Methods
Factor 3: Q2, Q3 ! Research Productivity

Quality of the factor model

Beyond the RMSR mentioned above, a technique widely used in sociology or psychology is the reliability
(Damásio, 2012). The reliability of a factor structure may be obtained by several criteria. Among other
criteria presented in the literature, the calculation of the level of internal consistency by Cronbach's alpha
(𝛼) is the most used method in cross-sectional studies - when measurements are performed in a single
moment (Sijtsma, 2009).
Cronbach's alpha coefficient measures the degree to which the items in an array of data are correlated.
Generally, the obtained index varies between 0 and 1. A commonly accepted rule for describing internal
consistency using Cronbach's alpha is:

• 0 to 0.49 unacceptable
• 0.50 to 0.59 poor
• 0.60 to 0.69 questionable
 136

• 0.70 to 0.79 acceptable

• 0.80 to 0.89 good
• from 0.9 to 1 excellent

Cronbach's alpha is influenced by the correlation values of the items and the number of evaluated items.
Therefore, factors with a few items tend to have lower Cronbach’s alpha and a matrix with high inter-item
correlations tend to have a high Cronbach's alpha.

Remembering the retained factor of the previous subsection:

Factor 1: Q1, Q6, Q7, Q9, Q10
Factor 2: Q4, Q5, Q8
Factor 3: Q2, Q3

The reliability analysis for the first factor is present below.

In R
### Internal consistency
Code

# PC1 (Q1, Q6, Q7, Q9, Q10)

library (psych)
alpha(survey[c("Q1", "Q6", "Q7", "Q9", "Q10")])
Reliability analysis
Call: alpha(x = survey[c("Q1", "Q6", "Q7", "Q9", "Q10")])

raw_alpha [Link] G6(smc) average_r S/N ase mean sd

0.71 0.71 0.67 0.32 2.4 0.053 3.7 0.76

lower alpha upper 95% confidence boundaries

0.61 0.71 0.81

Reliability if an item is dropped:

raw_alpha [Link] G6(smc) average_r S/N alpha se
Q1 0.62 0.62 0.55 0.29 1.6 0.069
Q6 0.66 0.66 0.61 0.33 2.0 0.064
Q7 0.69 0.68 0.64 0.35 2.2 0.062
Output

Q9 0.63 0.63 0.57 0.30 1.7 0.067

Q10 0.69 0.68 0.64 0.35 2.2 0.061

Item statistics
n raw.r std.r [Link] [Link] mean sd
Q1 200 0.76 0.74 0.68 0.56 3.5 1.21
Q6 200 0.67 0.67 0.54 0.45 3.4 1.08
Q7 200 0.60 0.63 0.47 0.40 4.0 0.95
Q9 200 0.73 0.72 0.63 0.52 3.8 1.19
Q10 200 0.63 0.63 0.47 0.39 3.9 1.13

Non missing response frequency for each item

1 2 3 4 5 miss
Q1 0.08 0.14 0.20 0.34 0.24 0
Q6 0.06 0.16 0.26 0.40 0.13 0
Q7 0.02 0.06 0.18 0.40 0.35 0
Q9 0.05 0.10 0.19 0.28 0.38 0
Q10 0.04 0.10 0.14 0.34 0.37 0
Table 91

In Python
 137

### Internal consistency

# Cronbach alphas function

def CronbachAlpha(itemscores):
itemscores = [Link](itemscores)
itemvars = [Link](axis=1, ddof=1)
Code

tscores = [Link](axis=0)
nitems = len(itemscores)

return nitems / (nitems-1.) * (1 - [Link]() / [Link](ddof=1))

# PC1 (Q1, Q6, Q7, Q9, Q10)

CronbachAlpha([Link](survey_data[[0,5,6,8,9]].transpose()))
Output

...: CronbachAlpha([Link](survey_data[[0,5,6,8,9]].transpose()))
Out[27]: 0.70830804374937695

Table 92

Regarding the 1st factor, the results show that α = 0.71 (raw_alpha), i.e., an acceptable value. The values
of the “reliability if an item is dropped” show a lower alpha value for all variables of this factor. This
means that all of them are contributing positively to the internal consistency of the factor. Hence, it can be
concluded that the first factor is well defined.

The analysis of the second factor is present below.

In R
### Internal consistency
Code

# PC2 (Q4, Q5, Q8)

library (psych)
alpha(survey[c("Q4", "Q5", "Q8")])
Reliability analysis
Call: alpha(x = survey [c("Q4", "Q5", "Q8")])

raw_alpha [Link] G6(smc) average_r S/N ase mean sd

0.66 0.66 0.57 0.39 1.9 0.077 3.3 0.85

lower alpha upper 95% confidence boundaries

0.51 0.66 0.81

Reliability if an item is dropped:

raw_alpha [Link] G6(smc) average_r S/N alpha se
Output

Q4 0.60 0.61 0.43 0.43 1.53 0.11

Q5 0.47 0.48 0.31 0.31 0.91 0.12
Q8 0.60 0.61 0.43 0.43 1.53 0.11

Item statistics
n raw.r std.r [Link] [Link] mean sd
Q4 200 0.72 0.75 0.55 0.44 3.2 0.96
Q5 200 0.82 0.81 0.66 0.53 3.6 1.19
Q8 200 0.77 0.75 0.55 0.45 3.1 1.13

Non missing response frequency for each item

1 2 3 4 5 miss
Q4 0.03 0.20 0.40 0.28 0.09 0
Q5 0.06 0.16 0.18 0.34 0.26 0
Q8 0.08 0.26 0.24 0.33 0.10 0
Table 93

In Python
 138

### Internal consistency

Code

# PC2 (Q4, Q5, Q8)

CronbachAlpha([Link](survey_data[[3,4,7]].transpose()))
Output

CronbachAlpha([Link](survey_data[[3,4,7]].transpose()))
Out[28]: 0.65970835814273876

Table 94

Regarding the second factor, an alpha value of 0.66 is presented. Additionally, if some item/variable of
this factor is dropped, the value of alpha decreases. Thus, it can be concluded that second factor is well
defined.

The analysis of the third factor is present below.

In R
### Internal consistency
Code

# PC3 (Q2, Q3)

library (psych)
alpha(survey[c("Q2", "Q3")])
Reliability analysis
Call: alpha(x = survey [c("Q2", "Q3")])

raw_alpha [Link] G6(smc) average_r S/N ase mean sd

0.51 0.52 0.35 0.35 1.1 0.12 3.9 0.77

lower alpha upper 95% confidence boundaries

0.27 0.51 0.74

Reliability if an item is dropped:

Output

raw_alpha [Link] G6(smc) average_r S/N alpha se

Q2 0.35 0.35 0.12 0.35 NA NA
Q3 0.35 0.35 0.12 0.35 NA NA

Item statistics
n raw.r std.r [Link] [Link] mean sd
Q2 200 0.78 0.82 0.48 0.35 3.9 0.83
Q3 200 0.86 0.82 0.48 0.35 3.9 1.04

Non missing response frequency for each item

1 2 3 4 5 miss
Q2 0.00 0.06 0.21 0.48 0.25 0
Q3 0.02 0.10 0.18 0.39 0.31 0
Table 95

In Python
### Internal consistency
Code

# PC3 (Q2, Q3)

CronbachAlpha([Link](survey_data[[1,2]].transpose()))
Output

CronbachAlpha([Link](survey_data[[1,2]].transpose()))
Out[29]: 0.50738200972962688

Table 96

Finally, the third factor has an alpha value of 0.51. Despite a little value of reliability, if some
item/variable is dropped, the alpha values decreases. Again, it can be concluded that the third factor is
also well defined.
 139

Factor Analysis vs. Principal Component Analysis

In factor analysis, the different assumption about the communalities is reflected in a different correlation
matrix as compared to the one used in the principal component analysis. Since in principal component
analysis all communalities are initially 1, the diagonal of the correlation matrix only contains unities. In
factor analysis, the initial communalities are not assumed to be 1. They are estimated (most frequently) by
taking the squared multiple correlations of the variables with other variables (Rietveld and Van Hout,
1993). These estimated communalities are then represented on the diagonal of the correlation matrix,
from which the eigenvalues will be determined, and the factors will be retained. After extraction of the
factors, new communalities can be calculated, which will be represented in a reproduced correlation
matrix (Kootstra, 2004).
The difference between factor analysis and principal component analysis is crucial in interpreting the
factor loadings: by squaring the factor loading of a variable, the amount of variance accounted by that
variable is obtained. However, in factor analysis, it is already initially assumed that the variables do not
account for 100% of the variance. Thus, as Rietveld & Van Hout (1993) state, “although the loading
patterns of the factors extracted by the two methods do not differ substantially, their respective amounts
of explained variance does!”

CONCLUSIONS
Factor analysis is an important task to execute a quantity reduction of variables. This is particularly useful
when there are many variables in our data. Thus, this is an important research area that should be
carefully studied by the data analyst. The central concepts that were presented in this chapter are:

• Bartlett Sphericity test

• KMO measure
• Retained factors
o Kaiser criterion
o Scree plot
o Variance explained criteria
• Factor analysis methods
o Principal component analysis
o Principal axis of factor extraction
o Maximum likelihood
• Factor rotations
• Internal consistency with Cronbach’s alpha

REFERENCES

Abdi, H. (2003). Factor rotations in factor analyses. Encyclopedia for Research Methods for the Social
Sciences. Sage: Thousand Oaks, CA, 792-795.

Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1 (2),
pp. 245-276.

Damásio, B. F. (2012). Uso da análise fatorial exploratória em psicologia. Avaliação psicológica, 11(2),
213-228.
View publication stats

140

Hayton, J.C., Allen, D.G. and Scarpello,V. (2004). Factor Retention Decisions in Exploratory Factor
Analysis: A Tutorial on Parallel Analysis. Organizational Research Methods, 7 (2), pp. 191-205 .

Kaiser, H.F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23,
pp.187-200.

Kootstra, G. (2004). Exploratory factor analysis. Unpublished Paper.

[Link] rema-stats-meth-seminar/[Link]

Ledesma, R. D. and Valero-Mora, p. (2007). Determining the Number of Factors to Retain in EFA: an
easy-to-use computer program for carrying out Parallel Analysis. Practical Assessment, Research &
Evaluation, 12 (2).

Marôco, J. (2011). Análise Estatística com o SPSS Statistics. 5th Edition. Pero Pinheiro: Report Number,
pp. 7-61.

Rietveld, T., and Van Hout, R. (1993). Statistical techniques for the study of language and language
behaviour. Walter de Gruyter.

Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha.
Psychometrika, 74(1), 107-120.