IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 25, NO.

10, OCTOBER 2014 1779

Sparse Alignment for Robust Tensor

Learning
Zhihui Lai, Wai Keung Wong, Yong Xu, Member, IEEE, Cairong Zhao, and Mingming Sun

Abstract— Multilinear/tensor extensions of manifold learning I. I NTRODUCTION

based algorithms have been widely used in computer vision and
pattern recognition. This paper first provides a systematic analy-
sis of the multilinear extensions for the most popular methods by
using alignment techniques, thereby obtaining a general tensor
I N RECENT years, high-order tensor learning methods
have been widely used in the fields of computer vision,
pattern recognition, and machine learning to deal with the
alignment framework. From this framework, it is easy to show curse of dimensionality problem. The classical dimension-
that the manifold learning based tensor learning methods are ality reduction method principal component analysis (PCA)
intrinsically different from the alignment techniques. Based on
the alignment framework, a robust tensor learning method called [1] was first extended to second-order cases (i.e., 2-D PCA
sparse tensor alignment (STA) is then proposed for unsupervised (2-DPCA) [2] and generalized low-rank approximations of
tensor feature extraction. Different from the existing tensor matrices (GLRAM) [3]), and then to the high-order case,
learning methods, L 1 - and L 2 -norms are introduced to enhance (i.e., multilinear PCA (MPCA) [4] and its uncorrelated varia-
the robustness in the alignment step of the STA. The advantage tion [5]). Similarly, linear discriminant analysis (LDA) [6] was
of the proposed technique is that the difficulty in selecting the
size of the local neighborhood can be avoided in the manifold also extended to 2-D LDA (2-DLDA) [7], [8] and multilinear
learning based tensor feature extraction algorithms. Although discriminant analysis (MDA) [9]. By using the differential
STA is an unsupervised learning method, the sparsity encodes scatter discriminant criterion (DSDC) [10], Tao et al. [11]
the discriminative information in the alignment step and provides proposed the general tensor discriminant analysis (GTDA) for
the robustness of STA. Extensive experiments on the well-known gait recognition. With the maximum margin criterion (MMC)
image databases as well as action and hand gesture databases
by encoding object images as tensors demonstrate that the [12], Laplacian bidirectional MMC (LBMMC) [13] and tensor
proposed STA algorithm gives the most competitive performance MMC (TMMC) [14] were proposed for object recognition.
when compared with the tensor-based unsupervised learning The high-order tensor-based methods performed better than
methods. the classical ones in feature extraction and classification.
Index Terms— Feature extraction, local alignment, manifold However, the above methods only use the global structure
learning, sparse representation, tensor learning. information of the dataset. Results from manifold learning
methods developed in the past decade show that the local
Manuscript received December 15, 2012; revised October 10, 2013;
accepted December 8, 2013. Date of publication January 13, 2014; date of geometric structure is more important than the global structure
current version September 15, 2014. This work was supported in part by the since the high-dimensional data lies on the low-dimensional
Natural Science Foundation of China under Grant 61203376, Grant 61375012, manifold. The representative manifold learning methods
Grant 61203247, Grant 61005005, Grant 61071179, Grant 61125305, Grant
61170077, Grant 61362031, Grant 61332011, Grant 61370163, and Grant include locally linear embedding (LLE) [15], ISOMAP [16],
61263032, in part by the General Research Fund of Research Grants Council Laplacian eigenmaps (LE) [17] and local tangent space align-
of Hong Kong under Project 531708, in part by the China Postdoctoral Science ment (LTSA) [18], and so on. All of these nonlinear manifold
Foundation under Project 2012M510958 and Project 2013T60370, in part by
the Guangdong Natural Science Foundation under Project S2012040007289, learning methods suffer from the out-of-sample problem [19],
and in part by the Shenzhen Municipal Science and Technology Innovation and one of the simplest but frequently used technique is to
Council under Grant JC201005260122A, Grant JCYJ20120613153352732, learn the explicit linear mappings of the corresponding nonlin-
Grant JCYJ20120613134843060, and Grant JCYJ20130329152024199.
Z. Lai is with the Bio-Computing Research Center, Shenzhen Graduate ear manifold learning methods. Therefore, locality preserving
School, Harbin Institute of Technology, Shenzhen 518055, China, and also projections (LPP) [20], the linearization of LE, neighborhood-
with the College of Computer Science and Software Engineering, Shenzhen preserving embedding (NPE) [21] and orthogonal neighbor-
University, Shenzhen 518055, China (e-mail: lai_zhi_hui@[Link]).
W. K. Wong is with the Institute of Textiles and Clothing, The Hong hood preserving projections (ONPP) [22], the linearization
Kong Polytechnic University, Hong Kong, and also with Shenzhen Research of LEE, the linear LTSA (LLTSA) [23] and its supervised
Institute, Shenzhen 518055, China (e-mail: [Link]@[Link]). variations [24], [25], the linearization of LTSA were proposed
Y. Xu is with the Bio-Computing Research Center, Shenzhen Graduate
School, Harbin Institute of Technology, Shenzhen 518055, China (e-mail: for dimensionality reduction. Recently, Rozza et al. [26]
yongxu@[Link]). proposed the truncated isotropic PCA classifier (T-IPCAC) for
C. Zhao is with the Department of Computer Science and Technology, feature extraction and classifier design.
Tongji University, Shanghai 201804, China (e-mail: zhaocairong@[Link]).
M. Sun is with the Department of Computer Science and Technology, Since these linear dimensionality reduction methods cannot
Nanjing University of Science and Technology, Nanjing 210094, China deal with the high-order tensor data, some of these methods
(e-mail: sunmingming@[Link]). were further extended to be multilinear cases, and many
Color versions of one or more of the figures in this paper are available
online at [Link] tensor-based and manifold learning based methods were pro-
Digital Object Identifier 10.1109/TNNLS.2013.2295717 posed by using higher order tensor decomposition [27]–[29].
2162-237X © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See [Link] for more information.
1780 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 25, NO. 10, OCTOBER 2014

Within the past 10 years, there has been great interest in high- (i.e., e, h, x etc.) denote vectors, bold uppercase letters (i.e.,
order tensor feature extraction, and the tensor-based methods A, B, C, etc.) denote matrices, and the Lucida calligraphy
have been popular in computer vision and pattern recogni- Italic letters, (i.e. X , Y) denote the tensors.
tion [30]–[33]. For example, He et al. [34] proposed tensor It is assumed that the training samples are represented as
subspace analysis (TSA) for second-order learning. Dai and the nth order tensor {Xi ∈ R m 1 ×m 2 ×···×m n , i = 1, 2, . . . , N},
Yeung [35] proposed tensor LPP (TLPP), tensor NPE (TNPE), where N denotes the total number of training samples.
and tensor LDE (TLDE). Yan et al. [36] proposed the marginal Definition 1: The mode-k flattening of the nth-order tensor
Fisher analysis (MFA) and graph embedding framework for X ∈ R m 1 ×m 2 ×···×m n (i = 1, 2, . . . , N) into matrix X(k) ∈
dimensionality reduction from the viewpoint of graph con- R m k × i=k m i , i.e. X(k) ⇐k X , is defined as Xi(k) = Xii ,i2 ,...,in ,
n n k,j
struction, in which some classical methods can be included. j = 1 + l=1,l =k (i l − 1) o=l+1,o =k m o .
Recently, Liu and Ruan [37] proposed orthogonal tensor NPE Definition 2: The mode-k product of tensor X with

(OTNPE) for facial expression recognition. By integrating the matrix U ∈ R m k ×m k isdefined as Y = X ×k U, where
mk
manifold learning and the MDA methods, discriminant locally Yi1 ,...,ik−1 ,i,ik+1 ,...,in = j =1 Xi1 ,...,ik−1 , j,ik+1 ,...,in Ui, j ( j =
linear embedding (DLLE) [38] and some variations such as 
1, . . . , m k ).
those in [39]–[42] were proposed for face recognition, gait The common properties of the tensor learning methods aim
recognition, action recognition, etc. (For more details, see to obtain a set of projection matrices {Ui ∈ R m i ×di , di ≤ m i ,
the latest survey of multilinear subspace learning for tensor i = 1, 2, . . . , n} and map the original high-order tensor data
data [43].) In addition, the tensor voting methods [44], [45] into a low-order tensor space, as
also used the tensor representation to perform the dimensional-
ity estimation, manifold learning, and function approximation. Yi = Xi ×1 U1 ×2 U2 · · · ×n Un . (1)
However, until now, a systematic analysis on the intrinsic
relationship among these tensor learning methods and design- Different tensor learning methods use different strategies to
ing a robust method for tensor learning have not been done. learn the projection matrices for feature extraction. With the
Therefore, this paper proposes to use the alignment techniques above preparations, some popular tensor learning methods
to unify the tensor learning methods and design a robust can be unified into a general framework which provides
tensor alignment method which integrates L 1 - and L 2 -norms the comprehensive understanding on different tensor feature
for sparse alignment. The contributions of this paper are as extraction methods.
follows. First, this paper proposes a general framework for
tensor learning and a concrete method called sparse tensor
alignment (STA) for feature extraction. Second, it provides a B. Proposed Tensor Alignment Technique and its Models
comprehensive analysis and comparison on some of the most Concerning the tensor learning methods, since the tensor Xi
representative tensor learning methods and puts them into a is unfolded into a large size matrix Xi(k) for computing, we
unified framework by using the tensor alignment techniques. only need to give the alignment method about the unfolded
Therefore, this framework leads us to understand the common matrix.
properties and intrinsic differences in existing tensor learning Let X̂i(k) = [Xi(k) , Xi(k)
1
, Xi(k)
2
, . . . , Xi(k)
K
] be the matrix con-
algorithms. Based on the unified framework summed up in (k)
taining Xi and its K unfolding nearest neighbors tensors.
this paper, a novel tensor learning method using the L 1 - and The projection matrix Uk maps the unfolding tensor into a
L 2 -norms penalty is proposed for robust tensor learning. Thus, (k) (k)
low-dimensional subspace: Uk : Xi → Yi . Let Li be the
it is natural for the proposed STA to avoid the difficulty in local alignment matrix of size (K + 1) × (K + 1) designed
selecting the neighborhood size in the manifold learning based for different tensor learning algorithms, and then the local
tensor learning methods. alignment optimization problem is formed
The rest of this paper is organized as follows. In Section II, a
 
systematic analysis on the tensor learning methods is provided. min tr Ŷi(k) (Li ⊗ Ik )Ŷi(k)T (2)
In Section III, a robust tensor alignment method is presented
and used for tensor learning. Experiments are carried out to where ⊗ denotes the Kronecker product of matrices and
(k) (k) (k) (k)
evaluate the proposed tensor learning method in Section IV, Ŷi = [Yi , Yi1 , . . . , Yi K ] be the local coordinate. The
and conclusions are given in Section V. selection matrix Si with the size of N × (K + 1) is defined as

II. T ENSOR A LIGNMENT T ECHNIQUES 1, if p = fi {q}
In this section, some basic multilinear notations, definitions (Si )pq = (3)
0, otherwise
and operations similar to those in [9] and [38] are briefly
reviewed at first and then the tensor alignment representation where fi = {i, i 1 , i 2 , . . . , i K } denotes the set of indices
of the most representative methods is presented. Thus a unified for the i th alignment matrix formed by Xi(k) (or tensor
tensor learning framework is obtained. Xi ) and its K unfolding nearest neighbors tensors. Let
(k) (k) (k)
Y(k) = [Y1 , Y2 , . . . , Y N ] be the global coordinates; then
A. Multilinear Algebras we have
In this paper, lowercase and uppercase italic letters
(k)
(i.e. i, j, N, etc.) denote scalars, bold lowercase letter Ŷi = Y(k) (Si ⊗ Ik ). (4)
LAI et al.: SPARSE ALIGNMENT FOR ROBUST TENSOR LEARNING 1781

Then (2) can be rewritten as of MPCA is

(k)
min tr(Ŷi (Li ⊗ Ik )Ŷi
(k)T
) min tr(UiT S(k)
t Ui )

= min tr(Y(k) (Si ⊗ Ik )(Li ⊗ Ik )(SiT ⊗ Ik )Y(k)T ) = min tr(UkT (Xi(k) − X̄(k) )(Xi(k) − X̄(k) )T Uk )
i

= min tr(Yik (Si Li SiT ⊗ Ik )Yi(k)T ). (5) N
1  (k)
N−1
= min tr (Yi − Yi(k) )(Yi(k) − Yi(k) )T
N2 j j
By summing up all the alignments together, the whole i=1 j =1

N   
alignment can be obtained as 1 (k) N −1
= min tr Ŷi ⊗ Ik
 (k) (k)T N 2 −e N−1
min tr(Ŷi (Si Li SiT ⊗ Ik )Ŷi ) i=1
 
i
 (k) N −1
T
= min tr
(k)T
Ŷi (Si Li SiT ⊗ Ik )Ŷi × Ŷi(k) ⊗ Ik
i −e N−1
= min tr(Y(k) (L ⊗ Ik )Y(k)T ) (6) N    T
1 N −1 N −1
 = min tr 2 Ŷi(k) ⊗ Ik Ŷi(k)T
N −e N−1 −e N−1
where L = T
i Si Li Si
is the alignment matrix [18], which i=1
can be obtained by the iterative procedure as 
N
= min tr(Ŷi(k) (LMPCA
i ⊗ Ik )Ŷi(k)T ) (11)
L(fi , fi ) ← L(fi , fi ) + Li (7) i=1

where Yi(k) ( j = 1, 2, . . . , N − 1) are the rest unfolded

with the initialization L = 0. j

Let La and Lb be some kinds of alignment matrices by dif- tensors of Yi(k) , X̄(k) is the unfolded mean tensor, and
ferent methods. If the linear transformation Yi(k) = UkT Xi(k) is e N−1 = [1, 1, . . . , 1]T with N − 1 elements, Ŷi(k) =
(k) (k) (k)
considered, then the following optimization model is obtained: [Yi , Yi1 , . . . , Yi K ] and K = N − 1
   T
min tr(Y(k)(L⊗Ik)Y(k)T)= min tr(UkT X(k) (La ⊗Ik)X(k)TUk) N −1 N −1
(i ) LMPCA = ⊗ Ik
s.t. UkT X(k) (Lb ⊗ Ik )X(k)T Uk = Idk .
i −e N−1 −e N−1
(8) and

LMPCA = LMPCA .
Or, if only one alignment matrix La is used and Y(k)
is i i
uniquely determined, the constraint Y(k) Y(k)T = Idk can be Therefore, MPCA can be viewed as a global tensor align-
imposed and then the optimization model is obtained as ment method since Ŷi(k) contains all the unfolded tensors. And
 model (ii) represents the optimization model of MPCA with
min tr(UkT X(k) (La ⊗ Ik )X(k)T Uk ) L = i LMPCA .
(9) i
s.t. UkT X(k) X(k)T Uk = Idk . The key points of T-IPCAC [26] are the recovery of the
residuals and the Fisher subspace estimation. Since T-IPCAS
Specifically, (9) is the special case of model (i) with Lb = I N . uses labeled data to estimate the Fisher subspace, it can also
In addition, one can alternatively impose the following be included in the MDA set. One of the key steps in T-IPCAC
orthogonal constraint and obtain another model is to compute the d eigenvectors corresponding to the d largest
 eigenvalues of the sample covariance matrix St on the vector
min tr(UkT X(k) (La ⊗ Ik )X(k)T Uk ) space instead of higher order tensor space, where the definition
(ii ) (10)
s.t. UkT Uk = Idk . of St and its alignment are as follows:

1  T   1 
N N
  T−IPCAC 
These two models can be solved by using the Lagrangian St = XX = X I X = XL i X
multiplier method and their solutions can be obtained by N N2
i=1 i=1
using generalized or standard eigenvalue decomposition, (12)
respectively. Since there are not closed-form solutions for 
tensor subspace learning methods, the iterative strategy is where X = [x1 , x2 , . . . , x N ] denotes the vector-based sample
usually used for computing the local optimal solutions. As matrix, and LT−IPCAC
i = (1/N 2 )I, and I is the N × N identity
can be seen from the following sections, these two mod- matrix. This indicates that the whitening step in T-IPCAC is
els are the basic forms of the tensor subspace learning a global alignment with the identity matrix.
methods.
D. Alignment for TLPP
TLPP preserves the local neighborhood relationship of
C. Alignment for MPCA and T-IPCAC
the tensors. Similar to LPP, TLPP first constructs the local
MPCA maximizes the trace of the total scatter matrix of the neighborhood matrix Wi j = exp(− X i − X j 2 /t) if X j is
unfolded tensors in the projected subspace. The basic model one of the K nearest neighbors of X i ; otherwise 0, and t
1782 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 25, NO. 10, OCTOBER 2014

is the tuning parameter. The objective function of TLPP is of TNPE is defined as

defined as 2

N 
K

  min Yi(k) − Mi, j Yi(k)

(k) (k) 2 j
min (Yi − Y j ) Wi j i=1 j =1
i j   
 −1  
N

N 
K
(k) (k) (k) (k) = min (k)
tr Ŷi −1 Mi,:
T
⊗ Ik Ŷi(k)T
= min (Yi − Yl )(Yi − Yl )T W̄il Mi,:
i=1
i=1 l=1
⎛⎡ ⎤ 
N
(k) (k)
N (Y − Yi1 )T = min tr(Ŷi(k) (LTNPE ⊗ Ik )Ŷi(k)T ) (15)
⎜⎢ i ⎥ i
= min tr ⎝⎣ ... ⎦ i=1
(k) (k)
i=1 (Yi − Yil )T
⎞ where Ŷi(k) = [Yi(k) , Yi(k)
1
, . . . , Yi(k)
K
], i.e., Ŷi(k) only contains
(k)
⎟ the Yi and its K nearest neighbor tensor unfolded matrices.
× [Yi(k) − Yi(k)
1
, . . . , Yi(k) − Yi(k)
l
]diag(W̄i,: )⎠ It can be found that TNPE is different from MPCA. The
essential difference is in the alignment matrices. TLPP uses

N   T    the local alignment method to construct LTNPE i .
(k) −e K (k)T
= min tr Ŷi diag(W̄i,: ) −eTK I K ⊗ Ik Ŷi
IK
i=1 F. Alignment for MDA
N
(k) (k)T
= min tr(Ŷi (LTLPP ⊗ Ik )Ŷi ) (13) MDA aims to find the multilinear subspaces that can min-
i
i=1
imize the trace of the within-class unfolded tensor scatter
matrix S(k)
w and maximize the trace of the between-class
(k)
  unfolded tensor scatter Sb .
−eTK   (k)
where LTLPP
i = diag(W̄i,: ) −eTK I K ⊗ Ik , and I K For the mode-k within-class tensor scatter matrix Sw ,
IK we have
is the K × K identity matrix, Ŷik = [Yik , Yik1 , . . . , YikK ],
min tr(S(k)
w )
e K = [1, 1, . . . , 1]T with K elements, and W̄il =
2
exp(− χi − χl /t) (i.e., matrix W̄ only contains nonzero 
C 
Ni
j (k) j (k)
elements in W̄). = (Yi − Ȳi(k) )(Yi − Ȳi(k) )T
In addition, TLPP has the following constraint which can i=1 j =1
also be represented by using the alignment technique: ⎛ ⎞⎛ ⎞T
 Ni −1 i −1
1 ⎝
N N
(k) (k) (k) (k)
= min tr (Yi − Yi j )⎠⎝ (Yi − Yi j )⎠
 
N  i=1
N2
j =1 j =1
tr Yi(k) Yi(k)T Di j = tr Yik (D ⊗ Ik ) Yi(k)T = 1 (14) ⎛ ⎛  T ⎞ ⎞
i N   Ni − 1
1 (k) Ni − 1 (k)T
i=1
= min tr ⎝ 2 Ŷi ⎝ −e N −1 ⊗ Ik⎠ Ŷi ⎠
N i −e N−1
i=1
where the
 diagonal elements Dii of matrix D is defined as 
Dii = 
N
1 (k)  w 
j Wi j . Equation (14) can be viewed as the single = min tr Ŷ Li ⊗ Ik Ŷi(k)T (16)
(k) (k) N2 i
alignment with weight Dii since matrix Ŷi = Yi only i=1
contains one element (i.e., the unfolded tensor of UkT Xi(k) ). (k)
Therefore, both parts of TLPP can be represented by using where Ȳi denotes the mean value of the mode-k flattening
the alignment technique. of the tensor samples in the i th class, C is the number of
j (k)
It should be noted that the tensor version of the graph classes, Ni is the number of the tensors in the i th [Link]
embedding framework proposed in [36] can also be repre- is the j th tensor in the i th class, e Ni −1 = [1, 1, . . . , 1]T with
sented and concluded in the tensor alignment framework with Ni − 1 elements and Ŷi(k) = [Yi(k) , Yi(k) 1
, . . . , Yi(k)
N −1
]
i
the same way as TLPP. To avoid repetition, it is omitted in   T
this paper. Ni − 1 Ni − 1
Lw
i = .
−e Ni −1 −e N−1
(k)
For the mode-k between-class tensor scatter matrix Sb ,
E. Alignment for TNPE we have

C
TNPE preserves the local linear reconstruction coefficients max tr(S(k)
b ) = Ni (Ȳi(k) − Ȳ(k) )(Ȳi(k) − Ȳ(k) )T
of tensors in the low-dimensional subspace. Suppose the ⎛ i=1 ⎞
coefficient matrix M (of size N × K ) is obtained in the C  (k)
C−1  (k)
C−1
1 (k) (k)
same way as in LLE, and M only contains the reconstruction = max tr ⎝ Ni 2 (Ȳi − Ȳi j ) (Ȳi − Ȳi j ) ⎠
T

coefficients (zero elements are not included). The cost function C

i=1 j =1 j =1
LAI et al.: SPARSE ALIGNMENT FOR ROBUST TENSOR LEARNING 1783

   T

C
Ni (k) C − 1 C−1 (k)T alignment matrices and preserve it in the low-dimensional
= max tr Ŷ ⊗ Ik Ŷi
C2 i −eC−1 −eC−1 subspace, therefore the manifold properties can be maintained.
i=1
 For the supervised tensor methods based on LPP or NPE, the
 Ni (k) ! b "
C
(k)T only difference from TLPP or TNPE lies in the construction of
= max tr Ŷ i L i ⊗ Ik Ŷi (17)
C2 the local neighborhood graphs W or M using the label infor-
i=1
mation. Thus, the corresponding supervised tensor learning
where Ȳ(k) denotes the mean value of the mode-k methods, such as those in [38]–[42], have the same alignment
flattening of the tensor samples of all the training samples. methods as in TLPP or TNPE. Therefore, this paper does not
(k)
Ȳi j ( j = 1, . . . , C − 1) is the mean unfolded tensor of the discuss it in detail except for the representative example MDA.
different classes form Ȳi(k) , and Table I summarizes the details of the tensor learning methods
using the proposed tensor alignment framework.
  T
C −1 C−1 Although the manifold learning based tensor learning meth-
Lbi =
−eC−1 −eC−1 ods usually outperform the globality-based methods such as
MPCA and MLDA, the neighborhood size K in the manifold
eC−1 = [1, 1, . . . , 1]T with C − 1 elements and learning based tensor subspace learning methods is difficult to
(k) (k) (k) (k)
Ŷi = [Ȳi , Ȳi1 , . . . , ȲiC−1 ]. decide in application. Moreover, since the tensor data usually
(k) contains large quantities of information redundancy and noise,
As can be seen from (16) and (17), Sw is aligned by the
(k) designing a robust method for alignment becomes crucial but
samples within each class, and Sb is aligned by the unfolded
matrices of the sample mean tensor of different classes. The has yet to be explored. In this paper, we introduce the recently
objective function of MDA can be constructed by using the proposed sparse representation for robust alignment in the next
model (i). section.

III. S PARSE T ENSOR A LIGNMENT

G. Alignment for Tensor Voting
In this section, a new multilinear dimensionality reduction
Tensor voting (TV) was proposed in [45] for dimensionality
technique called STA is developed as a special application of
estimation, manifold learning, and function approximation.
the above tensor alignment framework.
The key operation in the voting process is to compute the
voting accumulator through the local neighboring points. This
step can be viewed as the local alignment with the eigenvectors A. Motivation of Sparse Tensor Alignment
in a special form. For xi , the voting accumulator of its Sparse representation has been widely used in signal
neighborhood point x j can be expressed as processing, image processing, feature extraction, and pattern
recognition. Wright et al. [46] proposed the use of sparse
j ← L j + (λ1 − λ2 )Svote (xi , x j , ê1 ) + λ N Bvote (xi , x j )
LTV TV
representation for robust face recognition, Qiao et al. [47]

N−1 proposed sparsity-preserving projections (SPPs) for feature
+ (λd −λd−1 )Vvote (xi , x j , Te,d
i
) (18) extraction, and Cheng et al. [48] used the L 1 -graph for image
d=2 data clustering and subspace learning. As demonstrated in
where λk s are the eigenvalues corresponding to the voter; [47] and [48], the graphs constructed by the L 1 -norm have
Svote (xi , x j , ê1 ) denote the stick vote from xi to x j with ê1 the advantages of greater robustness to data noise, automatic
being the normal at xi ; Bvote (xi , x j ) denotes the ball vote from sparsity and adaptive neighborhood for individual datum.
xi to x j ; Vvote (xi , x j , Te,d
i ) denotes the vote from the generic Another important advantage is that sparse representation
i
tensor, and Te,d denotes the elementary voting tensor with d has the potential discriminative ability since most nonzero
equal nonzero eigenvalues. The readers are referred to [45] for elements are located on the samples in the same class as
more details. the represented sample [46]–[48]. Thus, these advantages can
Comparing (18) with (7), we can find that the tensor voting be naturally fused to tensor learning if the L 1 -norm is used
method in [45] has a procedure similar to the tensor alignment in the tensor representation in constructing the alignment
framework and thus it can be included in the proposed matrices. However, only using the L 1 -norm penalty such as
framework. in LASSO [49] has its limitation as indicated in [50]: if there
is a group of variables among which the pairwise correlations
are very high, LASSO tends to select any one variable from the
H. Discussions on the Tensor Learning Models group and does not consider which one is selected. Fortunately,
It can be seen from above sections that the tensor learning it is known that combining the L 1 - and L 2 -norm penalty can
methods are different in terms of constructing the alignment result in grouping effectiveness in regression and thus enhance
matrices L∗i and Ŷik . MPCA and MDA use the global align- the prediction accuracy by using the elastic net [50] which
ment techniques, but TLPP and TNPE use local alignment overcomes the limitation of only using the L 1 -norm penalty.
techniques. One of the limitations of MPCA and MDA is In short, it is expected that the elastic net is used to group
that the alignment matrices cannot reflect the local geometric a set of sparse coefficients to construct the sparse alignment
structure of a tensor dataset. However, TNPE and TLPP use matrices, in which the sparse representation information or the
the local geometric structure information in constructing the potential discriminative information is encoded to enhance the
1784 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 25, NO. 10, OCTOBER 2014

TABLE I
S UMMARY OF THE A LGORITHMS


discriminative ability in an unsupervised manner. Therefore, it to zero (implying that the xi is removed from X), and the
is reasonable to integrate these advantages to design a more elements h i, j ( j = i ) denote the contribution of each x j to
robust and effective tensor learning method. As a result, we reconstruct xi ; e is an N-dimensional vector of all 1s. Then
first introduce sparse representation and the SPP algorithm in the optimal solution, denoted as h̃i , is used to construct
the next section, and then STA is proposed. the following objective function which aims to preserve the
optimal weight vector h̃i
B. Sparse Representation and SPP 
N
 2  T
The goal of sparse representation is to represent the high- UT xi − UT Xh̃i = tr (UT X(I − H̃)(I − H̃)T X U)
 i=1
dimensional vector x as few entries of X = [x1 , . . . , x N ] as
(22)
possible. This can be formally expressed as follows:
 where H̃ = [H̃1 , h̃2 , . . . , H̃ N ]. The optimal projections of SPP
min h 0 s.t. x = Xh (19) are the eigenvectors corresponding to the smaller eigenvalues
h
of the following generalized eigenvalue problem:
where h ∈ R N is the coefficient vector and h 0
 T T  T
is the L 0 -norm which is equal to the number of nonzero X(I − H̃)(I − H̃) X U = XX U. (23)
components in h. Unfortunately, this problem is not convex,
and finding the sparse solution is NP-hard. It has been shown
that, if the solution of (19) is sparse enough, this difficulty can C. Efficient Method for Computing the Sparse Coefficients
be bypassed by convexizing the problem [51] and solving Since SPP only focuses on the vector-based sparse represen-
 tation problem using the L 1 -norm, in this section, the sparse
min h 1 s.t. x = Xh. (20) representation for tensor data combining the L 1 -and L 2 -norm
h
penalty is introduced. First, in order to obtain the optimal
SPP takes the advantage of L 1 -norm sparse representation
sparse representation coefficients, the tensor representation of
and preserves such reconstructive weights for dimensionality
the following L 1 - and L 2 -norm penalty optimization problem
reduction. For each xi , SPP first solves the following L 1 -norm
should be solved
minimization problem: ⎛ ⎞
2

⎜  # #⎟
min hi 1 s.t. xi = Xhi , 1 = eT hi (21) H∗ = arg min ⎝ X i − Hi j X j +α Hi,: +β #Hi,: #⎠(∀i )
2
H j, j  =i
where hi = [h i,1 , . . . , h i,i−1 , 0, h i,i+1 , . . . , h i,N
]T is an
N-dimensional vector in which the i th element is equal (24)
LAI et al.: SPARSE ALIGNMENT FOR ROBUST TENSOR LEARNING 1785

⎡
⎛ ⎞⎛ ⎞T ⎤
where the N × N matrix H is the representation coefficient  
⎢
⎝ xi − ⎥
matrix satisfying diag(H) = 0 (this is similar to the H̃ in = tr⎣ Hi j x j ⎠⎝xi − Hi j x j ⎠ T⎦
SPP), and Hi,: denotes the i th row vector, |·| denotes the L 1 - j, j  =i j, j  =i
norm of vector Hi,: , the coefficient α ≥ 0 is a parameter to # #
+α Hi,: + β #Hi,: #
2
control the amounts of shrinkage, and β is the L 1 -norm term ⎡ ⎤
⎛ ⎞⎛ ⎞T
coefficient. Because of the nature of the L 1 -norm penalty,  
⎢
⎝ xi − ⎥
some coefficients are shrunk exactly to zeros if β is large = tr⎣ Hi j x j ⎠⎝xi − Hi j x j ⎠ I⎦
enough. The difference for learning the reconstruction matrix j, j  =i j, j  =i
between the STA and SPP is that STA uses the L 1 - and # #
+α Hi,: + β #Hi,: #
2
L 2 -norm penalty which can result in sparsity and improving ⎡
the grouping effectiveness in regression [50] in unsupervised ⎛ ⎞⎛ ⎞T⎤
⎢   ⎥
manner. However, it is impossible to directly solve the above ⎝ xi −
= tr⎣ Hi j x j ⎠⎝xi − Hi j x j ⎠ ⎦
optimization problem with tensor representation. Fortunately, j, j  =i j, j  =i
it is easy to obtain the following proposition from Definition 1. # #
+ β #Hi,: #
2
Proposition 1: The optimization problem of (24) is equiva- +α Hi,:
lent to the following optimization problem: 2
⎛ ⎞  # #
+ β #Hi,: #.
2
2 = xi − Hi j x j + α Hi,:
⎜  # #⎟
H∗ = arg min ⎝ xi− Hi j x j +α Hi,: +β #Hi,: #⎠(∀i )
2 j, j  =i 2
H j, j  =i 2 Therefore, the optimal sparse reconstruction coefficients are
(25) invariant when xi s are projected to the low-dimensional sub-
space A.
where xi denotes the high-dimensional vector concatenated by The theorem indicates that the sparse representation coef-
(k)
the columns of matrix Xi (or tensor X i ) for any mode k. ficients can be efficiently computed in a low-dimensional
Therefore, one can solve the N optimization problem (25) subspace spanned by the xi s instead of being in the orig-
to obtain sparse matrix H by using the elastic net algorithm inal high-dimensional space. Therefore, the computational
[50]. However, since xi is a very high-dimensional vector, complexity can be greatly reduced in solving the sets of
directly solving (25) is also time consuming. Fortunately, the optimization problem (26).
following theorem can guarantee the equivalence of the sparse
representation coefficients, which can be computed efficiently.
Theorem 1: Suppose xi s are the independent random vec-
tors, for any unitary matrix  = [A Ac ] where span(A) = D. Sparse Tensor Alignment Algorithm
span(x1 , . . . , x N ) and Ac is the complement of A, the follow- Once the optimal sparse coefficient matrix H is obtained,
ing optimization problem (26) has the same solution as (25) it can be incorporated into the tensor alignment framework,
⎛ ⎞ in which the sparse representation coefficients are preserved.
2
⎜  # #⎟ Thus a novel unsupervised tensor dimensionality reduction
H∗ = arg min⎝ AT xi− Hi j AT x j +α Hi,: +β #Hi,: #⎠.
2
method called STA is obtained.
H j, j  =i 2 The objective function of STA is defined as
(26) 2

N
(k)

N
(k)
Proof: Since span(A) = span(x1 , . . . , x N ) and Ac is the min Yi − Hi, j Y j
orthogonal complement of A, then T  = T = I and i=1 j  =i, j =1
AcT xi = 0. We have 2
2 
N 
N
1
 # # = min ( Yi(k) − Hi, j Y(k)
j )
+ β #Hi,: #
2
A T xi − Hi j A T x j + α Hi,: N
i=1 j =1
j, j  =i 2 2
2 
N 
N
1
 # # = min ( ei − Hi, j )Y(k)
+β #Hi,: #
2 j
= [A Ac ]T xi − Hi j [A Ac ]T x j +α Hi,: N
i=1 j =1
j, j  =i 2

N  
2 1 1
 # # = min tr Ŷk ( ei − Hi,: )( ei − Hi,: )T ⊗ Ik Ŷ(k)T
+ β #Hi,: #
2 N N
=  T xi − Hi j  T x j + α Hi,: i=1
j, j  =i
⎡ ⎛ ⎞⎛
2
⎞T ⎤ 
N
= min tr(Ŷ(k) (LSTA ⊗ Ik )Ŷ(k)T )
⎢   ⎥
i
= tr ⎣T⎝xi − Hi j x j⎠⎝xi − Hi j x j ⎠ ⎦ ! i=1
! " "
j, j  =i j, j  =i = min tr Ŷ(k) (I − H)(I − H)T ⊗ Ik Ŷ(k)T
# #
+ β #Hi,: # = min tr(UkT X(k) (LSTA ⊗ Ik )X(k)T Uk )
2
+α Hi,:
1786 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 25, NO. 10, OCTOBER 2014

TABLE II
STA A LGORITHM P ROCEDURES

(k) (k) (k)

where Ŷ(k) = [Y1 , Y2 , . . . , Y N ], LSTA
i = (1/Nei − Hi,: ) From the above analysis and the literature [3]–[5], [9], [11],
(1/Nei − Hi,: )T and LSTA = (I − H)(I − H)T , and the N- [14], [30]–[33], and [38]–[43], we can conclude that when
dimensional vector ei = [0, 0, . . . , 0, 1, 0, ...0], i.e., only i th the data is a second- or high-order tensor, the tensor-based
element is 1. learning methods can improve computational efficiency and
By using the model (ii), the whole optimization model of avoid small sample size problem, thereby obtaining better
STA is obtained as follows: performances than the vector-based learning methods in this

min tr(UkT X(k) (LSTA ⊗ Ik )X(k)T Uk ) case. However, since the tensor-based learning models are
(27) the nonconvex optimization problems, they cannot obtain the
s.t. UkT X(k) X(k)T Uk = Idk .
global optimal solution.
For each mode k, the optimal projection matrix of STA can
be obtained by solving the following eigen equation: IV. E XPERIMENT
In this section, a set of experiments are presented to evaluate
X(k) (LSTA ⊗ Ik )X(k)T Uk = X(k) X(k)T Uk . (28)
the proposed STA, the baseline method (nearest classifier
Similar to other tensor learning methods, the optimal pro- on the original data), and other unsupervised algorithms,
jection matrices of STA have no closed-form solutions. How- i.e., MPCA, TLPP, TNPE and ONPP, for recognition tasks,
ever, the suboptimal solutions can be obtained by iteratively including second-order tensor (image matrix) in face/objective
optimizing different projection matrices while fixing the other recognition and high-order tensor (3-D matrix data) in action
projection matrices. The details of the STA algorithm are recognition. The Yale face database was used to explore the
shown in Table II. robustness of STA with the variations in expressions, illumina-
tion, block subtraction, and noise. The COIL-20 database was
E. Computational Complexity Analysis used to test the robustness of STA for pose variations in noise
For simplicity, we assume that m 1 = m 2 = · · · = m n = m and block subtraction. The FERET face database was used to
and the total number of training samples N is compara- test the robustness of STA with variations in face expression
ble in magnitude to the feature dimension m n . Usually, the and lighting conditions. The Weizmann database was used
computational complexity of multilinear extension methods is to test the performance of STA in high-order learning. The
less than that of vector-based methods on higher tensor data. nearest neighbor classifier with Euclidean distance was used in
For example, the complexity of PCA is O(m 3n ). The total all the experiments. Section IV-F summarizes the experimental
complexity of MPCA is O(t ((n + 1)Nnm n+1 + nm 3 )), where results.
t denotes the number of iterations in the outside loop.
In STA, solving the elastic net to get the coefficient matrix A. Robustness Test on Yale Face Database
needs O(m n N 2 J ), where J denotes the number of nonzero The Yale face database [52] ([Link]
elements and usually is a very small number. Thus it can projects/yalefaces/[Link]) contains 165 images of
be rewritten as O(m n N 2 ). Computing the scatter matrices in 15 individuals (each person providing 11 different images)
(27) needs O(n Nm n+1 ) (upper bounded). Solving (27) needs with various facial expressions and lighting conditions. In our
O(m 3 ) and the tensor projection needs O(Nm n+1 ). Therefore, experiments, each image was manually cropped and resized
the total complexity of STA is O(m n N 2 + t (n Nm n+1 + to 50 × 40 pixels. In order to test the robustness of the algo-
Nm n+1 + m 3 )). rithms, some areas of images were first replaced by a randomly
LAI et al.: SPARSE ALIGNMENT FOR ROBUST TENSOR LEARNING 1787

Fig. 1. Processed sample images of one person from the Yale face database.

TABLE III
AVERAGE R ECOGNITION R ATES (P ERCENT ), S TANDARD E RROR ,
AND THE B EST PARAMETER VALUE OF F IVE M ETHODS
ON THE YALE FACE D ATABASE

Fig. 2. (a) Average recognition rates (%) versus the number of dimensions
located square block of size 5 × 5 or 10 × 10. Then Gaussian on the Yale face database. (b) Recognition rates (%) versus the variation of
noise was added to the two groups of occluded images by α and the number of dimension on STA algorithm.
using the MATLAB code “a = awgn (a, 1, 35),” where
“a” denotes the image matrix. Fig. 1 shows the occluded
sample images (block size 10 × 10) of one person used in
the experiments.
In the experiments, six images of each individual were
randomly selected and used as training set, and half of the
remaining images as test and validation set, respectively. The
experiments were independently performed 10 times and the
average recognition results of the test set were calculated. For
Fig. 3. Processed sample images from the COIL-20 image database.
each run, the validation set was used for parameter selection
(i.e., the local neighbor size K and the optimal subspace As can be seen form Table III and Fig. 2, STA obtains
dimensions) in MPCA, TLPP, TNPE, and ONPP. When using the best recognition rates in the two groups of experiments,
the elastic net, the optimal parameter α is selected from which shows the robustness for block subtraction and noise
{0.001,0.01,…,10 000}. The parameter β can be automatically when there are variations in expressions and illumination.
determined since the elastic net algorithm could provide the
optimal solution path of β [50]. The average recognition rates
of each method and the corresponding best parameter values B. Robust Objective Recognition on COIL-20 Image Database
(in average) are shown in Table III. The recognition rate versus The COIL-20 database [53] ([Link]
the number of the dimensions is shown in Fig. 2(a), and the CAVE/software/softlib/[Link]) consists of 20 × 72 =
variation of the recognition rate versus the parameter α of a 1440 images of 20 objects where the images of each object
single run is shown in Fig. 2(b), which indicates that the STA were taken at pose intervals of 5° (i.e., 72 poses per object).
is robust to this parameter when it is large enough. Fig. 2(b) The original images were normalized to 128 × 128 pixels.
also shows that, when α = 0 (i.e., without the L 2 norm Each image was converted to a gray-scale image of
penalty), the STA usually is less effective than when using 32 × 32 pixels for computational efficiency in the experiments.
a suitable L 2 -norm penalty coefficient. Thus, the grouping The images were also preprocessed as in Section IV-A by
effectiveness in regression can enhance the performance of the using the 5 × 5 square block for occlusion. Some sample
algorithm. images of four objects are shown in Fig. 3.
1788 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 25, NO. 10, OCTOBER 2014

TABLE IV TABLE V
AVERAGE R ECOGNITION R ATES (P ERCENT ), S TANDARD AVERAGE R ECOGNITION R ATES (P ERCENT ), S TANDARD
E RROR , AND THE B EST PARAMETER VALUE OF F IVE E RROR , AND THE B EST PARAMETER VALUE OF F IVE
M ETHODS ON THE COIL-20 D ATABASE M ETHODS ON THE FERET D ATABASE

Fig. 4. Average recognition rates (%) versus the number of dimensions on

Fig. 6. Average recognition rates (%) versus the number of dimensions on
the COIL 20 database.
the FERET database.

Fig. 5. Sample images of one person on FERET face database.

In this database, the experiments were performed in the

same way as in Section IV-A. The recognition rates of each
method are shown in Table IV. The recognition rates versus the
Fig. 7. Key silhouettes of 10 actions from the Weizmann database. (a) Bend.
variations of the dimension are shown in Fig. 4. In Table IV (b) Jack. (c) Jump. (d) Pjump. (e) Run. (f) Side. (g) Skip. (h) Walk. (i) Wave1.
and Fig. 4, STA obtains the best recognition rate, which shows (j) Wave2.
the robustness for block subtraction and noise when there are
variations in rotations of the objectives.
as the same way in Section IV-A. Table V lists the recognition
rates of each method and Fig. 6 shows the variations of the
C. Experiments on FERET Face Database
recognition rates versus the dimensions. Again, STA performs
The FERET face database is a result of the FERET program, better than the other methods.
which was sponsored by the U.S. Department of Defense
[54]. It has become a standard database for testing and
evaluating state-of-the-art face recognition algorithms. The D. Experiments on Weizmann Action Database
proposed method was tested on a subset of the FERET The experiment was performed on the Weizmann data-
database. This subset includes 1400 images of 200 individuals base [55], which is a commonly used database for human
(each individual has seven images) and involves variations in action recognition. The 90 videos coming from 10 categories
facial expression, illumination, and pose. In the experiment, of actions included bending (bend), jacking (jack), jumping
the facial portion of each original image was automatically (jump), jumping in places (pjump), running (run), galloping
cropped based on the location of the eyes, and the cropped sideways (side), skipping (skip), walking (walk), single-hand
images were resized to 40 × 40 pixels. The sample images of waving (wave1), and both-hands waving (wave2), which were
one person are shown in Fig. 5. performed by nine subjects. The centered key silhouettes of
In the experiments, four images of each individual were each action are shown in Fig. 7.
randomly selected and used for training, and the remaining In order to represent the spatiotemporal feature of the
images were used for testing. The experiments were performed samples, 10 successive frames of each action were used to
LAI et al.: SPARSE ALIGNMENT FOR ROBUST TENSOR LEARNING 1789

is to extend previous works on spatiotemporal alignments by

incorporating manifold learning.

E. Experiments on Cambridge Hand Gesture Database

The Cambridge hand gesture database [57] consists of
900 image sequences of nine gesture classes, which are defined
by three primitive hand shapes and three primitive motions.
The objective of using this dataset is to classify different
shapes as well as different motions at a time. Each class
contains 100 image sequences (5 different illuminations × 10
arbitrary motions × 2 subjects). Each sequence was recorded
Fig. 8. Example of the bending action in the spatiotemporal domain from
Weizmann database.
in front of a fixed camera having roughly isolated gestures
in space and time. Thus, fairly large intraclass variations in
TABLE VI spatial and temporal alignment are reflected in the dataset.
AVERAGE R ECOGNITION R ATES (P ERCENT ), S TANDARD Some sample images of the nine different gesture classes
E RROR , AND THE B EST PARAMETER VALUE OF F IVE are shown in Fig. 10. The experimental procedures are the
M ETHODS ON THE W EIZMANN A CTION D ATABASE same as in the Weizmann action database. The recognition
rates of each method are listed in Table VII and the average
recognition rates (%) versus the number of dimensions are
shown in Fig. 9(c). It can be found that STA also outperforms
the other algorithms in hand gesture tensor feature extraction.

F. Discussion
Based on the experimental results shown in the above
sections, the following observations can be made.
1) Although the label information was not used in all
extract the temporal feature. Fig. 8 shows a tensor sample methods, STA obtained the best recognition rates. STA per-
of the bending action. Each centered frame was normalized formed better than the TNPE and OTNPE, which indicates
to the size of 32 × 24 pixels. Thus the tensor sample was that combining the L 1 - and L 2 -norms for sparse alignment
represented in the size of 32 × 24 × 10 pixels. It should be provides more discriminative information than local linear
noted that there are no overlapped frames in any two tensors reconstruction.
and the starting frames of the tensors are not normalized to the 2) STA was more robust than the other compared methods.
beginning frames of each action. Thus, the recognition tasks OTNPE outperformed MPCA, TLPP, and TNPE in higher
are difficult. Therefore, if one wants to get high recognition dimensional subspace, but it usually obtained low accuracies
accuracy, the methods used for feature extraction should be in the lower dimensional subspace. That is, with increasing
robust to the starting frames and the actions’ variations. number of dimensions, OTNPE is more effective than MPCA,
In the experiments, six action tensors of each category were TLPP, and TNPE in different cases.
randomly selected and used for training and the remaining 3) STA performed better than TLPP and TNPE, which
tensors were used for testing. The experimental procedures indicates that sparsity is more important than locality. In
were the same as in Section IV-A. The recognition rates of addition, TLPP and TNPE had almost the same perfor-
each method are listed in Table VI. The average recognition mance in action recognition, which indicates that only using
rates (%) versus the number of dimensions are shown in the L 2 -norm as a metric to measure the local geometric
Fig. 9(a). The variations of the average recognition rate versus structure cannot always improve performance.
the number of training sample of different methods are shown 4) In the experiments presented in Sections IV-A and IV-B,
in Fig. 9(b). It can be found that STA also outperforms the it was found that the local neighborhood graphs could not
other algorithms in action tensor feature extraction. Since explore the latent discriminant information for discrimination
many experimental details are quite different from each other when noise was added to the data, which gave rise to the lower
in different articles, all the results obtained by different recognition rates of TLPP, TNPE, and OTNPE. However, STA
algorithms in this paper are based on the same database did not introduce the local neighborhood parameter K and thus
and the same experimental background, and thus it is fair there was essential difference. In STA, the L 1 - and L 2 -norms
to compare them. When the number of training samples are combined together for grouping the reconstruction coeffi-
is slightly increased, the recognition rates of STA and the cients with sparse properties; thus the advantages of robustness
compared methods are above 90%. However, the superiority to data noise and the potential discriminative ability proven in
of the proposed STA over previous tensor-based subspace [46]–[48] are encoded in the representation coefficients, which
learning methods is still maintained. As indicated in [56], are preserved in the low-dimensional subspace. These are the
a possible improvement direction on the action recognition essential reasons for STA to achieve good performance.
1790 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 25, NO. 10, OCTOBER 2014

Fig. 9. (a) Variations of the average recognition rates (%) versus the number of dimensions on the Weizmann action database. (b) Average recognition rate
versus the number of training sample of different methods on the Weizmann action database. (c) Variations of the average recognition rates (%) versus the
number of dimensions on Cambridge hand gesture database.

L 2 -norm penalty. STA preserved the sparse tensor representa-

tion coefficients, which encoded the discriminative information
and robustness in the low-dimensional subspace. Experimental
results on five well-known databases showed the excellent
performance of STA against the state-of-the-art tensor learning
methods in face recognition, objective recognition, and action
recognition. It was shown that STA is robust to noise, block
Fig. 10. Some sample images on Cambridge hand gesture database. subtraction, rotation of the object, and starting frames of
different actions. For future research, we plan to enforce the
TABLE VII sparsity on the projection matrix/vector and investigate the
AVERAGE R ECOGNITION R ATES (P ERCENT ), S TANDARD E RROR , sparse projection learning methods for tensor recognition.
AND THE B EST PARAMETER VALUE OF F IVE M ETHODS
ON THE C AMBRIDGE H AND G ESTURE D ATABASE R EFERENCES
[1] M. Turk, “Eigenfaces for recognition,” J. Cognit. Neurosci., vol. 3, no. 1,
pp. 71–86, Jan. 1991.
[2] J. Yang, D. Zhang, A. F. Frangi, and J. Yang, “Two-dimensional PCA: A
new approach to appearance-based face representation and recognition,”
IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 1, pp. 131–137,
Jan. 2004.
[3] J. Ye, “Generalized low rank approximations of matrices,” Mach. Learn.,
vol. 61, nos. 1–3, pp. 167–191, 2005.
[4] H. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, “MPCA: Multilin-
ear principal component analysis of tensor objects,” IEEE Trans. Neural
Netw., vol. 19, no. 1, pp. 18–39, Jan. 2008.
[5] H. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, “Uncorrelated
5) From the experiments, we also found the limitation multilinear principal component analysis for unsupervised multilin-
of the STA algorithm. The properties of STA were very ear subspace learning,” IEEE Trans. Neural Netw., vol. 20, no. 11,
similar to those of the compared algorithms. However, since pp. 1820–1836, Nov. 2009.
[6] C. Fraley and A. E. Raftery, “Model-based clustering, discriminant
the coefficient matrix was obtained from elastic net which analysis and density estimation,” J. Amer. Statist. Assoc., vol. 97, no. 1,
selected the group of the most correlated samples, if the group pp. 611–631, 2002.
of coefficients corresponding to the correlated samples was [7] J. Yang, D. Zhang, X. Yong, and J. Yang, “Two-dimensional discriminant
transform for face recognition,” Pattern Recognit., vol. 38, no. 7,
distributed in different classes (i.e., elastic net cannot explore pp. 1125–1129, 2005.
more discriminant information than the local geometric struc- [8] M. Li and B. Yuan, “2D-LDA: A statistical linear discriminant analysis
ture in TLPP, TNPE), STA might not obtain higher recognition for image matrix,” Pattern Recognit. Lett., vol. 26, no. 5, pp. 527–532,
2005.
rate than TLPP or TNPE. However, this special case seldom [9] S. Yan, D. Xu, Q. Yang, L. Zhang, X. Tang, and H.-J. Zhang, “Mul-
happened. tilinear discriminant analysis for face recognition,” IEEE Trans. Image
Process., vol. 16, no. 1, pp. 212–220, Jan. 2007.
V. C ONCLUSION [10] K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed.
San Diego, CA, USA: Academic Press, 1990.
In this paper, we chose a set of tensor learning algorithms [11] D. Tao, X. Li, X. Wu, and S. J. Maybank, “General tensor discriminant
and unified them by using the alignment technique. As a result, analysis and Gabor features for gait recognition,” IEEE Trans. Pattern
a general tensor learning framework was obtained. By using Anal. Mach. Intell., vol. 29, no. 10, pp. 1700–1715, Oct. 2007.
[12] H. Li, T. Jiang, and K. Zhang, “Efficient and robust feature extraction by
this framework as a platform, STA was proposed to explore maximum margin criterion,” IEEE Trans. Neural Netw., vol. 17, no. 1,
the latent discriminative information by using the L 1 - and pp. 157–165, Jan. 2006.
LAI et al.: SPARSE ALIGNMENT FOR ROBUST TENSOR LEARNING 1791

[13] W. Yang, J. Wang, M. Ren, and J. Yang, “Feature extraction based on [39] W. Zhang, Z. Lin, and X. Tang, “Tensor linear Laplacian discrimination
Laplacian bidirectional maximum margin criterion,” Pattern Recognit., (TLLD) for feature extraction,” Pattern Recognit., vol. 42, no. 9,
vol. 42, no. 11, pp. 2327–2344, 2009. pp. 1941–1948, Sep. 2009.
[14] R.-X. Hu, W. Jia, D.-S. Huang, and Y.-K. Lei, “Maximum mar- [40] F. Wang and X. Wang, “Neighborhood discriminant tensor mapping,”
gin criterion with tensor representation,” Neurocomputing, vol. 73, Neurocomputing, vol. 72, nos. 7–9, pp. 2035–2039, Mar. 2009.
nos. 10–12, pp. 1541–1549, Jun. 2010. [41] Y. Liu, Y. Liu, and K. C. C. Chan, “Tensor distance based multilin-
[15] S. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by ear locality-preserved maximum information embedding,” IEEE Trans.
locally linear embedding,” Science, vol. 290, no. 22, pp. 2323–2326, Neural Netw., vol. 21, no. 11, pp. 1848–1854, Nov. 2010.
2000. [42] D. Xu, S. Yan, D. Tao, S. Lin, and H.-J. Zhang, “Marginal Fisher analysis
[16] J. B. Tenenbaum, V. de Silva, and J. C. Langford, “A global geometric and its variants for human gait recognition and content-based image
framework for nonlinear dimensionality reduction,” Science, vol. 290, retrieval,” IEEE Trans. Image Process., vol. 16, no. 11, pp. 2811–2821,
no. 22, pp. 2319–2323, 2000. Nov. 2007.
[17] M. Belkin and P. Niyogi, “Laplacian eigenmaps for dimensionality [43] H. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, “A survey of
reduction and data representation,” Neural Comput., vol. 15, no. 6, multilinear subspace learning for tensor data,” Pattern Recognit., vol. 44,
pp. 1373–1396, Jun. 2003. no. 7, pp. 1540–1551, Jul. 2011.
[18] Z. Zhang and H. Zha, “Principal manifolds and nonlinear dimensionality [44] G. Guy and G. Medioni, “Inferring global perceptual contours from local
reduction via tangent space alignment,” SIAM J. Sci. Comput., vol. 26, features,” Int. J. Comput. Vis., vol. 20, nos. 1–2, pp. 113–133, 1996.
no. 1, pp. 313–338, 2004. [45] P. Mordohai and G. Medioni, “Dimensionality estimation manifold
[19] Y. Bengio, J. F. Paiement, P. Vincent, O. Delalleau, N. Roux, and learning and function approximation using tensor voting,” J. Mach.
M. Ouimet, “Out-of-sample extensions for LLE, Isomap, MDS, Eigen- Learn. Res., vol. 11, no. 1, pp. 411–450, 2010.
maps, and spectral clustering,” in Proc. Adv. Neural Inf. Process. Syst., [46] J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust face
2004, pp. 1–8. recognition via sparse representation,” IEEE Trans. Pattern Anal. Mach.
[20] X. He and P. Niyogi, “Locality preserving projections,” in Proc. Adv. Intell., vol. 31, no. 2, pp. 210–227, Feb. 2009.
Neural Inf. Process. Syst., vol. 16. 2004, pp. 153–160. [47] L. Qiao, S. Chen, and X. Tan, “Sparsity preserving projections with
[21] X. He, D. Cai, S. Yan, and H.-J. Zhang, “Neighborhood preserving applications to face recognition,” Pattern Recognit., vol. 43, no. 1,
embedding,” in Proc. 10th IEEE Int. Conf. Comput. Vis., vol. 2, pp. 331–341, Jan. 2010.
Oct. 2005, pp. 1208–1213. [48] B. Cheng, J. Yang, S. Yan, Y. Fu, and T. S. Huang, “Learning with L1
[22] E. Kokiopoulou and Y. Saad, “Orthogonal neighborhood preserving graph for image analysis,” IEEE Trans. Image Process., vol. 19, no. 4,
projections: A projection-based dimensionality reduction technique,” pp. 858–866, Apr. 2010.
IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 12, pp. 2143–2156, [49] R. Tibshirani, “Regression shrinkage and selection via the lasso,”
Dec. 2007. J. R. Statist. Soc., vol. 58, no. 1, pp. 267–288, 1996.
[23] T. Zhang, J. Yang, D. Zhao, and X. Ge, “Linear local tangent space [50] H. Zou and T. Hastie, “Regularization and variable selection via the
alignment and application to face recognition,” Neurocomputing, vol. 70, elastic net,” J. R. Statist. Soc. Series B (Statist. Methodol.), vol. 67,
nos. 7–9, pp. 1547–1553, Mar. 2007. no. 2, pp. 301–320, 2005.
[24] Y. Li, D. Luo, and S. Liu, “Orthogonal discriminant linear local [51] R. Baraniuk, “A lecture on compressive sensing,” IEEE Signal Process.
tangent space alignment for face recognition,” Neurocomputing, vol. 72, Mag., vol. 24, no. 4, pp. 118–121, Jul. 2007.
nos. 4–6, pp. 1319–1323, Jan. 2009. [52] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriengman, “Eigenfaces
[25] T. Zhang, D. Tao, X. Li, and J. Yang, “Patch alignment for dimen- vs. Fisherfaces: Recognition using class specific linear projection,”
sionality reduction,” IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711–720,
pp. 1229–1313, Sep. 2009. Jul. 1997.
[26] A. Rozza, G. Lombardi, E. Casiraghi, and P. Campadelli, “Novel [53] S. A. Nene, S. K. Nayar, and H. Murase, “Columbia object image library
Fisher discriminant classifiers,” Pattern Recognit., vol. 45, no. 1, (COIL-20),” Dept. Comput. Sci., Columbia Univ., New York, NY, USA,
pp. 3725–3737, 2012. Tech. Rep. CUCS-005-96, 1996.
[27] T. G. Kolda, “Orthogonal tensor decompositions,” SIAM J. Matrix Anal. [54] P. J. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss, “The FERET
Appl., vol. 23, no. 1, pp. 243–255, 2001. evaluation methodology for face recognition algorithms,” IEEE Trans.
[28] L. Lathauwer, B. De Moor, and J. Vandewalle, “A multilinear singular Pattern Anal. Mach. Intell., vol. 22, no. 10, pp. 1090–1104, Oct. 2000.
value decomposition,” SIAM J. Matrix Anal. Appl., vol. 21, no. 4, [55] L. Gorelick, M. Blank, E. Shechtman, M. Irani, and R. Basri, “Actions
pp. 1253–1278, 2000. as space-time shapes,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 29,
[29] L. Lathauwer, B. De Moor, and J. Vandewalle, “On the best rank-1 and no. 12, pp. 2247–2253, Dec. 2007.
randk-(R1, R2, ...RN) approximation of high-order tensors,” SIAM J. [56] D. Gong and G. Medioni, “Dynamic manifold warping for view invariant
Matrix Anal. Appl., vol. 21, no. 4, pp. 1324–1342, 2000. action recognition,” in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011,
[30] M. A. O. Vasilescu and D. Terzopoulos, “Multilinear analysis of pp. 571–578.
image ensembles: Tensiofaces,” in Proc. Eur. Conf. Comput. Vis., 2002, [57] T.-K. Kin, S.-B. Wong, and R. Cipolla, “Tensor canonical correlation
pp. 447–460. analysis for action classification,” in Proc. IEEE CVPR, Jun. 2007,
[31] M. A. O. Vasilescu and D. Terzopoulos, “Multilinear image analysis for pp. 1–8.
facial recognition,” in Proc. 16th Int. Conf. Pattern Recognit., vol. 2.
2002, pp. 511–514.
[32] M. A. O. Vasilescu and D. Terzopoulos, “Multilinear subspace analysis
for image ensembles,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., Jun. 2003, pp. 93–99.
[33] S. Rana, W. Liu, M. Lazarescu, and S. Venkatesh, “A unified tensor
framework for face recognition,” Pattern Recognit., vol. 42, no. 11,
pp. 2850–2862, Nov. 2009.
[34] X. He, D. Cai, and P. Niyogi, “Tensor subspace analysis,” in Proc. Adv.
Neural Inf. Process. Syst., 2005, pp. 499–506.
[35] G. Dai and D. Yeung, “Tensor embedding methods,” in Proc. 21st AAAI
Conf. Artif. Intell., vol. 1. 2005, pp. 330–335.
[36] S. Yan, D. Xu, B. Zhang, H.-J. Zhang, Q. Yang, and S. Lin, “Graph Zhihui Lai received the B.S. degree in mathematics from South China
embedding and extensions: A general framework for dimensionality Normal University, Guangzhou, China, the M.S. degree from Jinan Univer-
reduction,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 1, sity, Guangzhou, China, and the Ph.D. degree in pattern recognition and
pp. 40–51, Jan. 2007. intelligence system from the Nanjing University of Science and Technology,
[37] S. Liu and Q. Ruan, “Orthogonal tensor neighborhood preserving Nanjing, China, in 2002, 2007, and 2011, respectively.
embedding for facial expression recognition,” Pattern Recognit., vol. 44, He has been a Research Associate and Post-Doctoral Fellow with The Hong
no. 7, pp. 1497–1513, Jul. 2011. Kong Polytechnic University, Hong Kong, from 2010 to 2013. His current
[38] X. Li, S. Lin, S. Yan, and D. Xu, “Discriminant locally linear embedding research interests include face recognition, image processing and content-
with high-order tensor data,” IEEE Trans. Syst., Man, Cybern. B, based image retrieval, pattern recognition, compressive sense, human vision
Cybern., vol. 38, no. 2, pp. 342–352, Apr. 2008. modelization, and applications in the fields of intelligent robot research.
1792 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 25, NO. 10, OCTOBER 2014

Wai Keung Wong received the Ph.D. degree from The Hong Kong Polytech- Cairong Zhao received the B.S. degree from Jilin University, Jilin, China,
nic University, Hong Kong. the M.S. degree from the Changchun Institute of Optics, Fine Mechanics and
He is an Associate Professor with The Hong Kong Polytechnic Uni- Physics, Chinese Academy of Sciences, Changchun, China, and the Ph.D.
versity. He has published more than 50 scientific articles in refereed jour- degree from the Nanjing University of Science and Technology, Nanjing,
nals, including the IEEE T RANSACTIONS ON N EURAL N ETWORKS AND China, in 2011, 2006, and 2003, respectively.
L EARNING S YSTEMS , Pattern Recognition, the International Journal of He is currently an Assistant Professor with Tongji University, Shanghai,
Production Economics, the European Journal of Operational Research, the China. He is the author of more than 15 scientific papers in pattern recognition
International Journal of Production Research, Computers in Industry, and and computer vision. His current research interests include face recognition,
the IEEE T RANSACTIONS ON S YSTEMS , M AN , AND C YBERNETICS . His feature extraction, and computer vision.
current research interests include artificial intelligence, pattern recognition,
and optimization of manufacturing scheduling, planning, and control.

Yong Xu (M’06) received the B.S. and M.S. degrees from the Air Force
Institute of Meteorology, Nanjing, China, in 1994 and 1997, respectively,
and the Ph.D. degree in pattern recognition and intelligence system from the
Nanjing University of Science and Technology, Nanjing, in 2005.
He was with the Shenzhen Graduate School, Harbin Institute of Technology Mingming Sun received the B.S. degree in mathematics from Xinjiang Uni-
(HIT), Harbin, China, from 2005 to 2007, as a Post-Doctoral Research Fellow. versity, Urumqi, China, in 2002, and the Ph.D. degree in pattern recognition
He is currently a Professor with the Shenzhen Graduate School, HIT. He was and intelligence systems from the Department of Computer Science, Nanjing
a Research Assistant Researcher with The Hong Kong Polytechnic University, University of Science and Technology (NUST), Nanjing, China, in 2007.
Hong Kong, from 2007 to 2008. He has published more than 40 scientific He is currently a Lecturer with the School of Computer Science and
papers. His current research interests include pattern recognition, biometrics, Technology, NUST. His current research interests include pattern recognition,
and machine learning. machine learning, and image processing.