This article has been accepted for inclusion in a future issue of this journal.

Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON CYBERNETICS 1

Cross-Lingual Knowledge Transferring by

Structural Correspondence and Space Transfer
Deqing Wang , Member, IEEE, Junjie Wu , Jingyuan Yang, Baoyu Jing,
Wenjie Zhang, Xiaonan He, and Hui Zhang

Abstract—The cross-lingual sentiment analysis (CLSA) aims to I. I NTRODUCTION

leverage label-rich resources in the source language to improve
ITH THE rapid development of the global retail
the models of a resource-scarce domain in the target lan-
guage, where monolingual approaches based on machine learning
usually suffer from the unavailability of sentiment knowledge.
W e-commerce market, consumers from various countries
often post online reviews in different languages to express
Recently, the transfer learning paradigm that can transfer sen- their “like” or “dislike” opinions of purchased products. These
timent knowledge from resource-rich languages, for example, reviews are deemed valuable to both businesses and consumers
English, to resource-poor languages, for example, Chinese, has
gained particular interest. Along this line, in this article, we to better understand the useful feedback and assessment.
propose semisupervised learning with SCL and space transfer However, researchers are often facing such a challenge when
(ssSCL-ST), a semisupervised transfer learning approach that dealing with reviews in various languages: although high-
makes use of structural correspondence learning as well as space quality and annotated English review corpora are usually
transfer for cross-lingual sentiment analysis. The key idea behind
ssSCL-ST, at a high level, is to explore the intrinsic sentiment
available, nonEnglish corpora (e.g., Chinese and Japanese) are
knowledge in the target-lingual domain and to reduce the loss of more difficult to acquire. Therefore, how to transfer cross-
valuable knowledge due to the knowledge transfer via semisuper- lingual knowledge from the source language domain to target
vised learning. ssSCL-ST also features in pivot set extension and language domain is the primary task of cross-lingual sen-
space transfer, which helps to enhance the efficiency of knowledge timent analysis (CLSA). The key idea behind CLSA is to
transfer and improve the classification accuracy in the target
language domain. Extensive experimental results demonstrate bridge the gap of vocabularies and/or semantics between the
the superiority of ssSCL-ST to the state-of-the-art approaches source and target language domains. Along this line, var-
without using any parallel corpora. ious methods based on multilingual sentiment lexicons or
Index Terms—Cross-lingual sentiment classification, space sentimental corpora [1]–[4]; machine translation [5]; transfer
transfer, structural correspondence learning (SCL). learning [6]–[9]; and feature representation learning [10], [11],
have been proposed for CLSA tasks.
For multilingual sentiment lexicons, a bilingual dictionary
or a translation oracle is often necessary when building subjec-
tivity lexicons [12]. However, the amount of bilingual lexicons
is limited and many words could not be contained. Moreover,
the multiword expression problem [13], [14] and the difficulty
Manuscript received June 30, 2019; revised June 22, 2020 and October
18, 2020; accepted January 3, 2021. This work was supported in part of lemmatizing also make it difficult to generate a proper sen-
by the National Key Research and Development Program of China under timent lexicon. In this article, we introduce a “one-to-many”
Grant 2019YFA0707204; in part by the NSFC under Grant 71725002, mapping extension strategy to address the multiword expres-
Grant 72031001, Grant 72021001, Grant U1836206, and Grant 61773361; and
in part by the National Information Security Plan under Grant 2018A033. This sion problem, which helps capture more sentiment words from
article was recommended by Associate Editor Y. Jin. (Corresponding author: two lingual domains.
Junjie Wu.) Machine translation-based approaches are widely used in
Deqing Wang, Wenjie Zhang, and Hui Zhang are with the School of
Computer Science, Beihang University, Beijing 100191, China (e-mail: CLSA tasks, which either employ machine translation or use
dqwang@[Link]). parallel corpora in the first step, and then apply the mono-
Junjie Wu is with the School of Economics and Management, Beihang lingual algorithms for text analysis, such as co-training [15]
University, Beijing 100191, China, also with the Beijing Advanced
Innovation Center for Big Data and Brain Computing, Beihang University, and mixture model [16]. The high-quality parallel corpora,
Beijing 100191, China, and also with the Beijing Key Laboratory of however, are often difficult to collect. Especially, for some
Emergency Support Simulation Technologies for City Operations, Beihang
University, Beijing 100191, China (e-mail: wujj@[Link]).
rare languages, it is even impossible to obtain the par-
Jingyuan Yang is with the School of Business, George Mason University, allel corpora and machine translation systems. Moreover,
Fairfax, VA 22030 USA (e-mail: jyang53@[Link]). the classification performance may be degraded severely
Baoyu Jing is with the Department of Computer Science, University of
Illinois at Urbana–Champaign, Champaign, IL 61801 USA.
by both the erroneous translation [15] and the difference
Xiaonan He is with the Department of Search, Baidu Inc., Beijing 100094, of the expression styles across language domains [11].
China. To address the above two limitations, our proposed
Color versions of one or more figures in this article are available at
[Link] method employs pivot features to connect two language
Digital Object Identifier 10.1109/TCYB.2021.3051005 domains.
2168-2267 
c 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See [Link] for more information.

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

2 IEEE TRANSACTIONS ON CYBERNETICS

Feature representation learning aims to induce a rea- target language domain from low-dimensional to high-
sonable feature representation between the source and the dimensional by selecting the most confidential instances
target language domains so as to reduce distributional dif- of unlabeled data and improves the classification accu-
ferences. Many methods based on topic model have been racy in the target language domain.
proposed for the CLSA task, for example, cross-cultural Latent 3) We conduct extensive experiments on two widely
Dirichlet Allocation (ccLDA) [17], topic correlation analysis adopted cross-lingual sentiment classification datasets
(TCA) [18], partially supervised cross-collection topic model with abundant state-of-the-art methods as benchmarks.
(PSCCLDA) [19], cross-lingual joint aspect/sentiment model Experimental results show that our approach signifi-
(CL-JAS) [20], and cross-lingual aspect and sentiment unifica- cantly outperforms some state-of-the-art methods with-
tion model (CL-ASUM) [10]. The drawback of these methods out using any parallel corpora or machine translation
lies in that new feature representations are often difficult to techniques.
explain, while the time complexity is generally too high. In The remainder of this article is organized as follows.
this article, we propose to transfer feature space between two Section II briefly reviews related work on sentiment anal-
language domains by a linear classifier, which is more suitable ysis. We then formally present our method in Section III.
for large-scale data. Sections IV and V provide experimental results on two
In this article, we focus on transfer learning approaches that datasets. We finally conclude our work in Section VI.
aim to transfer knowledge from rich-labeled resources in the
source language domain to a resource-scarce target language
domain [21]. It is well known that there are yet many unsolved
difficulties in applying transfer learning for CLSA tasks. For II. R ELATED W ORK
example, the noise injection by the machine translator [15] and In this section, we briefly review the recent sentiment clas-
the loss of valuable knowledge [11], [22] are deemed to be sification studies from the monolingual and the cross-lingual
inevitable during knowledge transferring steps. Prettenhofer perspectives, respectively.
and Stein [23] proposed the cross-lingual adaptation using The monolingual model is first proposed to analyze the emo-
structural correspondence learning (CL-SCL), which tried to tions or sentiment orientation from large-scale online reviews.
connect two feature spaces based on structural correspondence Generally, these approaches rely on two kinds of resources,
learning (SCL). The advantage of CL-SCL is that it requires that is: 1) sentiment lexicons and 2) corpora [28]–[32]. For
fewer linguistic resources, but its performance is critically lexicon-based models, we need to construct and apply different
affected by both the one-to-one mapping of “pivots” and the sentiment lexicons from datasets; for corpora-based models,
low dimensionality of feature space [7], [24]. we generally employ machine-learning techniques [28] to
As the objective of CLSA is to improve the performance predict sentiment polarities. For example, in [1] and [29],
of the classifier in the target language domain, the suitable they tried to identify the sentimental words or phrases and
implementation of the sentiment knowledge and unlabeled then determine the sentiment polarity by these features.
data could be very valuable for the target domain. To this Mullen and Collier [31] used support vector machines (SVMs)
end, we propose a fine framework, called semi supervised to fuse multiple sources of potentially related information,
learning with SCL and space transfer (ssSCL-ST). By inte- such as phrases, adjectives, and topics. To further improve sen-
grating SCL and space transfer simultaneously, our approach timent extraction, the aspect-opinion association structure was
could reduce the loss of some valuable information in the introduced and a “first clustering, then extracting” unsuper-
source language domain and utilize the internal information vised model was proposed to extract sentimental words [32].
in the target language domain effectively [25]. Specifically, The above methods have made some progress for monolingual
we first improve CL-SCL by “one-to-many mapping” pivot sentiment classification. However, they rely on high-quality-
extension to learn an initial classifier. Then, we transfer fea- extracted sentiment lexicons or corpora, and that makes them
ture spaces and select confidential examples from unlabeled difficult to transfer the learned sentiment knowledge from one
target-language data under the framework of semisupervised language to other ones.
learning [26], [27]. The learning process of ssSCL-ST is just Cross-lingual sentiment classification tasks become more
like a transfer chain since cross-lingual knowledge is trans- challenging than monolingual ones, since the target domain
ferred by selecting confidential instances and changing their has a different feature space with the source domain.
corresponding feature spaces. Finally, we learn a classifier Therefore, many cross-lingual models are proposed to address
from the high-dimensional feature space in the target-lingual the above problem from different ways [4], [33]–[37]. For
domain. example, Banea et al. [38], [39] employed machine trans-
Our main contributions are summarized as follows. lation to address cross-lingual sentiment classification tasks.
1) We propose a one-to-many mapping extension strategy of Moreover, bilingual parallel corpora and dictionaries are
the pivot set for traditional SCL, which helps to detect the also widely used resources for CLSA tasks. For instance,
bridging knowledge between the source language domain Mihalcea et al. [1] proposed to generate sentiment resources
and the target language domain more comprehensively. by a bilingual dictionary and parallel corpora and to train the
2) Then, we propose a fine semisupervised framework with sentence-level classification models. Wan [5], [15] proposed a
the help of space transfer and abundant unlabeled target- co-training approach for cross-lingual sentiment classification,
language data, which transfers feature spaces of the which applied automatic machine translation (e.g., Google

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

WANG et al.: CROSS-LINGUAL KNOWLEDGE TRANSFERRING 3

translate) and ensemble methods to jointly predict the senti- The above methods for cross-lingual sentiment classifica-
ment polarity of a review. Hajmohammadi et al. [40], [41] tion tasks need high-quality lexicons, machine translation, or
proposed to combine uncertainty-based active learning and bilingual parallel data. SCL [51], [52], which is related to the
self-training together or employ multiple source languages work done by Ando and Zhang [53], [54], however, needs
in a multiview semisupervised learning for CLSA problems. fewer lingual resources. Prettenhofer and Stein [6], [23] then
These machine translation-based methods are often affected put forward CL-SCL, a cross-lingual SCL method, to solve the
by the quality of translation or parallel corpora [15], [21], problem of cross-lingual sentiment classification. Specifically,
for example, the wrong translation or noise will decrease the they adopted the correlations between “pivot” and nonpivot
classification accuracy. Moreover, it is expensive and time con- features in corpora to obtain a low-dimensional projection
suming to acquire aligned parallel data in different languages. matrix, which are the transferred knowledge between two lan-
Therefore, it becomes more difficult when applying them for guages domains. Obviously, the above method does not need
some resource-scarce languages. any parallel corpora.
Since the topic model [42] can induce a latent feature repre-
sentation, there have been some cross-collection/lingual mod-
els in the literature. For coarse-grained models, ccLDA [17]
III. S EMISUPERVISED K NOWLEDGE T RANSFERRING BY
and PSCCLDA [19] encoded exact alignment by forcing spe-
S TRUCTURAL C ORRESPONDENCE AND S PACE T RANSFER
cific topics between two domains to share the same topic
indexes with common topics. TCA [18] utilized common A. Model Framework
topics to learn a projection matrix between specific topics For cross-lingual text classification tasks, given a source
of different domains. The disadvantage of the above meth- language domain DS and a target language domain DT , the
ods lies in that exact alignment restricts the representation feature spaces of two language domains are XS and XT ,
flexibility, which might lead to a sharp accuracy drop when respectively. Here, we assume that there is a nonoverlap-
the distributions of the source and target language domains ping region between XS and XT . Therefore, the objective of
are very different [43]. In other words, the assumption of our model is to learn a sentiment predictor fT for exam-
exact alignment is often violated [44] since different lan- ples in the target language domain by applying the labeled
guage domains usually differ in their underlying distributions. training ones in the source language domain. The key of
Moreover, coarse-grained models only learn document-level cross-language sentiment classification is to bridge the fea-
feature representations, which often fail to capture various ture spaces between HS = {XS , PS (·)} in the source-language
aspects in one real-life product review [45], [46], for example, domain and HT = {XT , PT (·)} in the target-language domain,
the screen, battery, and camera of an iPhone. For fine-grained where PS (·) and PT (·) are the data distributions in the two
models, CL-JAS [20] sampled topics from source and tar- domains, respectively.
get language domains by word-level translation, which might Traditional CL-SCL for CLSA task has the following
cause semantic drift and result in inaccurate topic-word distri- problems.
butions because of synonyms and polysemy. The latest CL-JST 1) The one-to-one mapping of “pivots” from the source
and CL-ASUM only achieved comparable performances to language domain to the target language domain. For
SVM [10]. The main concern of topic model-based methods example, Li et al. [7] extended SCL from one-to-one
is that the new feature representations are difficult to explain mapping to one-to-many mappings for pivot words by
explicitly. Moreover, the time complexity is too high to apply using large amounts of monolingual data and a small
for large-scale data. dictionary to learn distributed representation of words.
Transfer learning approaches aim to adopt learned knowl- 2) The low dimensionality problem of feature space, that
edge from the source language domain to improve the is, some valuable knowledge may be lost in the step of
performance of the target language domain. For example, projection from a high-dimensional feature space to the
some recent studies argued that the high-level concepts low-dimensional one [24].
helped model the difference of data distribution between two 3) Besides, the linear classifier of CL-SCL is also trained
domains and, thus, were more appropriate for cross-lingual under the low-dimensional feature space of the source
sentiment classification [25], [47], [48]. Specifically, these language domain [23], rather than the target language
methods assume that all data domains have the same set of domain. The above problems decrease the performance
shared/identical/alike concepts and distinct concepts, which of CL-SCL. In this article, we propose to integrate
are used as the bridge for knowledge transfer. For example, semisupervised learning with SCL and space transfer
dual transfer learning (DTL) [25] modeled the shared con- (ssSCL-ST) to solve the above problems.
cepts including the identical and alike concepts to establish To address the above problems, we proposed ssSCL-ST, as
the bridge between the source and the target domains. Triplex shown in Fig. 1. First, we propose a one-to-many mapping
transfer learning (Tri-TL) [48] built classifiers on the shared extension strategy of the pivot set (see details in Section III-B)
concepts among the identical concepts, alike concepts, and to detect the bridging knowledge between the source language
distinct concepts. Zhuang et al. [49] integrated a regularized domain and the target language domain more comprehensively.
manifold structure of the target domain to further enhance Tri- Second, CL-SCL is employed to learn the projection matrix θ .
TL. Hu et al. [50] proposed multiknowledge transfer from Third, we train a linear classifier under the low-dimensional
multisource domains to the target domain. feature space to predict the labels of unlabeled examples in

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON CYBERNETICS

unlabeled documents in both source language and target lan-

guage domains, and then learns a projection matrix θ based
on the occurrence of pivots between two domains. It means
pivots are bridging knowledge between source and target lan-
guage domains. However, in CL-SCL, pivots are mapped by
a “one-to-one” strategy, that is, each pivot in the source lan-
guage domain only has on the corresponding pivot in the target
language domain. In this article, we first improve CL-SCL by
the “one-to-many mapping” scheme, and then connect the two
domains and finally obtain the low-dimensional feature spaces
θ XS and θ XT .
Given a set of the labeled training data DS written in source
Fig. 1. Framework of ssSCL-ST. language S, DS comprises tuples of the form (x, y), where
x ∈ XS and y ∈ YS . In addition, we have access to unlabeled
documents D S,u and DT,u from both languages S and T. Let Du
denote DS,u DT,u . Thus, the key step for solving the cross-
lingual problem is about how to compute the projection matrix
θ from DS and DT,u . The original CL-SCL lacks robustness
and is seriously affected by the strict one-to-one mapping of
pivots between the source language domain and target lan-
guage domain [7], in which the word translation oracle maps
each pivot feature in the source language domain to exactly
only one word in the target language domain. In this article,
we propose a new extension strategy of pivot features, that
is, one-to-many mapping. Particularly, we apply a synonym
dictionary to extend pivot features across the source and the
target language domains. Through the one-to-many mapping,
the connection between two domains becomes denser, rather
Fig. 2. Transferring chain of feature spaces. than sparse.
Here, the brief steps of pivot extension and computation of
projection θ in our proposed method are described as follows.
the target language domain. Fourth, the semisupervised learn- One-to-Many Pivot Extension: First, we select a subset VP
ing technique is adopted to iteratively select most confidential from VS with the largest mutual information value in source
examples from these unlabeled examples in the target language language-labeled data DS , where |VP |  |VS | and VS is the
domain (see details in Section III-C). Finally, these confiden- vocabulary of DS . Here, pivots are some word pairs {wS , wT },
tial target language examples are trained with linear SVM where wS is the pivot word in the source language and wT is
as our final predictor for the test set in the target language the pivot word in the target language. Thus, we can obtain the
domain, through which we finish the space transfer from low- original pivot set P by using the Google Translator to translate
dimensional space to target domain-related high-dimensional each pivot in VP .
space. However, the original pivot set P is selected by one-
In the above framework, we implement feature space trans- to-one mapping from the source language domain and the
fer between two domains by a transferring chain, in which target language domain, which severely affects the prediction
semisupervised learning is used to select the most confiden- performance [7]. To overcome the above disadvantage of the
tial instances from target language unlabeled data through the original pivot set, in this article, we propose to further extend
transformation of feature spaces, as illustrated in Fig. 2. Note pivots by a one-to-many mapping strategy, that is, pivot set P
that the knowledge of labeled documents in the source lan- is extended with the help of a synonym dictionary.
guage is transferred from XS to XT through θ XS and θ XT , Particularly, for each wS ∈ P , it is extended to a set of
which are both lower dimensional feature spaces. Next, we synonyms SYwS by using WordNet.1 Through the extension,
employ the classifier trained in a high-dimensional feature we obtain |SYwS | synonyms of wS and combine SYwS with the
space of the target language domain to partially recover the lost translated word in the target language to construct the new
information due to the reduction of dimensionality of feature word pair. Finally, we obtain an extended pivot set P.
spaces. We will present the details of our ssSCL-ST approach Take the pivot word “wonderful” as an example, the
in the following sections. word pair {“wonderful,” } will be extended as
the following new word pair, that is, {“grand,” “fantas-
tic,” “marvelous,” “marvellous,” “wonderful,” “wondrous,”
B. Pivot Extension: One-to-Many Mapping “terrific,” “tremendous,” }. After that, each word
CL-SCL [6] is a cross-lingual model by applying SCL,
which first employs mutual information to select pivots from 1 [Link]

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

WANG et al.: CROSS-LINGUAL KNOWLEDGE TRANSFERRING 5

in the original pivot set P is extended as a syn- C. Space Transfer With Semisupervised Learning
onyms set. By pivot set extension, the connection between To not only restore the information loss during the knowl-
the source and the target language data becomes more edge transfer but also exploit the knowledge of unlabeled
steady. data in the target language domain, we propose to recover
Projection Matrix Learning: After determining the extended the feature space of the target language domain from a low-
pivot set, we will learn the correlations between each pivot dimensional space into a high-dimensional one by using the
{wS , wT } ∈ P and all other words w ∈ V\{wS , wT }, which can semisupervised learning technique [26], which selectively con-
be done by training classifiers that predict whether or not wS or structs a new target language training set from a large amount
wT occurs in a document based on the other words. Therefore, of related but unlabeled examples.
for each pivot pair pl ∈ P, we can construct a training set In CL-SCL, the unlabeled examples are only used once
Dl = {(MASK(x, pl ), IN(x, pl ))|x ∈ Du }, where MASK(x, pl ) to learn the projection θ . In contrast, we try to utilize the
returns a copy of x with the components associated with the unlabeled data and convert the cross-language classification
words in pl set to 0—this is equivalent to removing these into a semisupervised learning problem. Specifically, given the
words from the feature space, and IN(x, pl ) returns +1 if one source language data DS,u and target language unlabeled data
component of x associated with the words in pl is nonzero DT,u (note that DS,u is obtained from DS without considering
and −1 otherwise. label information), the feature matrix of documents set Du is
Then, for each Dl , we train a linear classifier characterized a block matrix
by the parameter vector wl to model the correlation between  
MS 0
the pivot {wS , wT } and the other words w ∈ / {wS , wT } in the Du = (2)
0 MT
documents by using the following equation:
 where matrix MS (|VS |×|DS,u |) represents the source language
 
wl = arg min L y, wT x + λ||w||2 (1) documents and matrix MT (|VT | × |DT,u |) represents the target
w∈R|V| (x,y)∈Dl language documents.
This first classifier we trained is used to predict the class
where |V| = |VS | + |VT | is the total number of features in labels of unlabeled target language instances by XT and the
source and target language domains, and L is a loss func- projection of θ , and it is defined as fS : {θ xi } → yi , xi ∈ XS
tion. Here, wl is the parameter of model for the lth pivot. and yi ∈ YS . A related point to consider is that the real fea-
Suppose we have m pivots, then we can align all param- ture space of the training set is θ ∗ (MS 0), which means
eters vectors into a |V| × m-dimensional parameter matrix: that the prediction of target language examples is implemented
W = [w1 |w2 | . . . |wm ]. In the projection step, we com- under the feature space θ ∗ (0 MT ). In CL-SCL, it adopts fS to
pute the singular value decomposition of matrix W (i.e., predict the labels of target language examples. However, the
[UDVT ] = SVD(W)), then we obtain a low-dimensional lin- performance of CL-SCL is not applicable for real-world appli-
ear approximation to the original feature space by selecting cations due to the loss of knowledge from a high-dimensional
top k left singular vectors (i.e., θ = UT [1:k,1:|V|] ). feature space to a low-dimensional one. To this end, in this
After we obtain projection matrix θ , for each labeled article, we propose to employ plenty of unlabeled exam-
instance xi in the source language domain, its original feature ples and semisupervised learning to iteratively construct target
vector xi can be projected as a new shared low-dimensional language datasets, which help take advantage of intrinsic sen-
feature vector θ xi . Then, a linear classifier fS (x) = sign(v∗T θ x) timent knowledge in the target language and replenish the lost
can be learned under the new low k-dimensional feature space, information of knowledge transfer.
where v∗ is the model parameters. Apparently, we first predict the label of each unlabeled
For the traditional CL-SCL approach, the classifier fS (x) is target language instance xj (1 ≤ j ≤ |DT,u |) under the
directly used to predict target language test data. fS (x), how- low-dimensional projection space θ xj with the classifier fS ,
ever, performs worse and is not applicable for the real-world and we obtain a new label space YT . Then, we merge the
application problems, because the information loss during instances and their labels with probability and select the top
knowledge transfer from high-dimensional space to a low N instances with the largest probabilities to construct a new
dimensional one has severely disrupted the performance of training dataset DT,L = {(x1 , y1 ), . . . , (xN , yN )} whose cor-
the a classifier in the target language domain. Besides, tradi- responding feature space is XT,L , and the second classifier
tional CL-SCL only employed the unlabeled data Du to learn fT,L {θ xi } → yi , xi ∈ XT , yi ∈ YT can be learned on DT,L .
the projection matrix, but it is worth noting that DT,u also Even though fT,L is trained under the low-dimensional projec-
provides a large amount of unlabeled data in the related tar- tion space θ XT , but it is worthy of noting that the feature space
get language. This can be used to improve the performance has been transferred from the source language θ XS to the tar-
of classifier by the semisupervised learning. To address the get language θ XT . In other words, we try to label some target
major defects of CL-SCL, we propose to employ semisu- language instances by label propagation between the source
pervised learning technique to improve the generalization and the target language domains.
of the prediction model, because we can select some con- The primary objective of our model is to transform a low-
fidential samples from DT,u to rectify the bias of training dimensional space θ XT in the source language domain into a
data. We will present semisupervised learning in the next high-dimensional space XT in the target language domain with
section. the help of semisupervised learning. Therefore, we employ

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON CYBERNETICS

Algorithm 1 ssSCL-ST CL-SCL to project feature spaces XS and XT from two lan-
Input: guages domains into a low-dimensional common feature space
Labeled data in source language DS XT,L . Second, we predict the labels of target language instances
Unlabeled data in source language DS,u with fS and obtain a target language-labeled dataset DT,L .
Unlabeled data in target language DT,u Third, we further train a classifier fT,L on DT,L under the low-
Parameters: m, k, λ, , N, r dimensional target language feature space. Fourth, we employ
Output: Linear SVM classifier fT the learned fT,L to repredict the labels of instances in DT,u
1. Pivots_Selection_Extension(Du , m) and obtain a new label space YT . We generate a new datasets
VS = MutualInformation(DS ) DT = {(x1 , y1 ), . . . , (xN∗r , yN∗r )} by selecting the top N ∗ r
P = {{wS , translate(wS )}, |wS ∈VS } most confidential instances from DT,u . Finally, we train fT on
P = {{SYwS , wT }|wS ∈P , wT ∈VT } DT as the final classifier for target language documents.
2. Pivot_Structure_Learning(Du , P) The complexity of ssSCL-ST comprises of three parts,
for l = 1 to m do namely, the complexity of structure learning of pivots, the
Dl = {(MASK(x, pl ), IN(x,
 pl ))|x ∈T Du } λ SVD of the dense matrix W, and the complexity of semisu-
wl = arg min w ∈ Rk L(y, w x) + 2 ||w||2 pervised learning. For the first part, we have to train a large
(x,y)∈Dl number of linear classifiers, in which the plain stochastic
end for
gradient descent (SGD) algorithm [55] and the learning rate
W = [w1 · · · wm ]
schedule from PEGASOS [56] are adopted, so the time com-
3. Compute_SVD(W, k)
plexity is about O(mTNV), where m is the number of pivot pair
UVT = SVD(W)
in P, T is the iterative steps of SGD in each linear classifier,
θ = UT [1:k,1:|V|]
and N and V are the numbers of instances and features, respec-
4. Train_Classifier fS : 
tively. However, we can adopt parallel learning algorithm to
v∗S = arg min v ∈ Rk L(y, vT θ xS ) + λ2 ||v||2
(x,y)∈DS improve the efficiency of learning linear classifiers. Second,
fS (x) = sign(v∗S T θ x) the SVD of dense parameter matrix W is the computational
5. Train_Classifier fT,L : bottleneck of our proposed model, but it can be computed
DT,L = InstanceSelection((y, more efficiently by setting negative entries in W as 0 to obtain
 v θ xT ),TN)
T
∗
vT,L = arg min v ∈ R k L(y, v θ xT ) + λ2 ||v||2 a sparse matrix [53], the time complexity is about O(Vm2 ).
(x,y)∈DT,L Finally, the complexity of semisupervised learning includes
fT,L (x) = sign(v∗T,L θ x) on DT,L
T
both the complexity of one linear classifier and one SVM clas-
6. Train_Classifier fT : sifier with linear kernel, the time complexity is about O(N  V  ),
DT = InstanceSelection((y, vT θ xT ), r ∗ N) where N  is the finally selected number of instances from target
Train SVM classifier: language unlabeled data and V  is the corresponding number
fT : xT → yT on DT of features for all N  instances.
Output: fT
IV. E XPERIMENTAL S ETTINGS
In this section, we provide the details of the experimen-
tal settings, datasets of our empirical evaluation, comparative
fT,L to repredict the labels for all instances in DT,u and
methods, and evaluation metrics.
obtain another label space YT . Then, we combine the instances
and the corresponding labels, and select the top r ∗ N most
A. Cross-Lingual Datasets
confidential instances from DT,u to generate a new dataset
DT = {(x1 , y1 ), . . . , (xN∗r , yN∗r )}, where xi ∈ XT and yi ∈ YT . In our experiments, we use a number of real-world text col-
In this phase, we choose the confidential instances by the ratio lections for the validation of our proposed approach. English
r of the total number of documents in the target language is considered as the source language while Chinese as the tar-
domain to get rid of the uncertain ones. Finally, we train a get language for the cross-language sentiment classification
linear SVM classifier fT : {xT } → yT on DT , which is used task. The datasets are about products reviews from Amazon2
for target language test documents. Note that other classifiers and iHerb,3 including customers’ reviews on products written
on the monolingual dataset can also be used for the last step. by English and Chinese. For each review, it has one positive
or negative sentiment label. Moreover, a bilingual dictionary
is employed for word translation and an English synonym
D. ssSCL-ST Algorithm dictionary is adopted for pivot extension.
The detailed pseudo code of ssSCL-ST is shown in The first dataset is the NLP&CC 2013 cross-language sen-
Algorithm 1. A linear classifier fS is trained based on the timent analysis data (in short, NLP&CC),4 which includes
CL-SCL method. During the process of computing the pro- source-language training, target-language unlabeled, and test
jection matrix θ , the pivot set is expended with a one-to-many set. There are three major product categories in NLP&CC,
mapping strategy. After that, we employ semisupervised learn- 2 [Link]
ing to build a multistage bridge from the source language 3 [Link]
space to target language space. Specifically, we first expand 4 [Link]

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

WANG et al.: CROSS-LINGUAL KNOWLEDGE TRANSFERRING 7

namely, BOOKS, DVD, and MUSIC. For each category, 4000 selected from unlabeled data and added into training set until
labeled English products reviews (train_EN) and over 10 000 all the unlabeled reviews are labeled.
unlabeled Chinese products reviews (train_un_CN) are col- Co-Training With NTD: Gui et al. [22] proposed an
lected from [Link]. We remove the labels of train_EN, improved co-training method, which identifies noises through
then we combine them with the remaining documents as the detection of negative transfers.
unlabeled data (Train_un_EN). For the testing dataset, 4000 CL-SCL: CL-SCL7 is proposed by Prettenhofer and
labeled Chinese reviews (test_CN) for each category are used Stein [23]. As suggested by the authors, we use the Google
to validate the performance of our model. Translate as the bilingual dictionary, and we set m = 600 and
The second dataset is crawled from iHerb. For some health k = 120 for a better performance, while the rest parameters
products, we collect 4000 English reviews and evenly divide are the same with CL-SCL (i.e., φ = 30 and λ = 10−5 ), which
them into positive and negative categories according to their achieves the best performance.
rating stars. Specifically, reviews with four or five stars belong Neural Network-Based Approaches: A number of deep
to the positive category, and those with one or two stars belong learning approaches have been proposed to address the cross-
to the negative category. Meanwhile, we also collect 6000 lingual sentiment classification tasks. We compared ssCL-SCL
Chinese reviews and divide them into the target language test with two state-of-the-art deep learning methods, that is, the
set (2000) and unlabeled dataset (4000). bilingual sentiment word embedding method (BSWE) [57]
For each dataset, we balance the labeled data according to and Wasserstein distance-guided representation learning (WD-
two orientations (POSITIVE and NEGATIVE). As each review GRL) [58]. BSWE first employed denoising autoencoder to
contains a summary and a text for detailed contents, we merge learn bilingual word embeddings, and then incorporated sen-
the contents of these two parts into one document d, while timent information of reviews into the bilingual embeddings.
each document d is represented as the feature vector x with a Because BSWE was also evaluated on the NLP&CC 2013
unigram model. For the Chinese text, we employ ICTCLAS5 dataset, the parameters of BSWE are initialized as the authors
(Zhang et al., 2003) to segment it. Then, the TF-IDF scheme suggested [57]. For the iHerb dataset, we tried different ini-
is used to compute the weight of each feature xi . tializations and reported the best results in our experimental
section. WD-GRL [58] is an adversarial learning model for
B. Comparative Methods domain adaptation tasks, which utilizes a neural network
to estimate the empirical Wasserstein distance between the
In the following experiments, we compare our ssSCL-ST
source and target samples and optimizes the feature extrac-
with some state-of-the-art baselines. It is worth noting that
tor network to minimize the estimated Wasserstein distance in
SVM is adopted as a basic classifier for its reported excellent
an adversarial manner. This implies the need for translating
performances in sentiment classification tasks.
the Chinese reviews into English ones before applying WD-
Machine Translation-Based Methods [MT-SVM(EN2CN)
GRL on NLP&CC 2013 and iHerb datasets. For WD-GRL,
and MT-SVM(CN2EN)]: For machine translation-based meth-
the network is designed with one hidden layer of 500 nodes,
ods, we often translate source/target language examples into
relu activation function, softmax output function, and fixed
target/source language ones by the Google Translator.6 Then,
learning rate 10−4 . Moreover, we reimplemented RBST-s and
the cross-lingual sentiment classification becomes a mono-
RNST-hp [8] and compared them with our proposed methods.
lingual classification. Specifically, for MT-SVM(EN2CN), we
Among the above comparative methods, MT-SVM is the
translate the English training set (train_EN) to Chinese texts
baseline method, and its prediction accuracy is easily affected
with correspondence labels, and an SVM classifier with a lin-
by the quality of the automatic translation. DTL and TriTL
ear kernel is trained on the translated training reviews. In
are two state-of-the-art transfer learning methods by applying
contract, for MT-SVM(CN2EN), the Chinese test set (test_CN)
non-negative matrix trifactorization. Both co-training and co-
is translated into English texts with correspondence labels.
training with NTD need the parallel corpora and also depend
DTL: DTL [25] learns marginal and conditional distribu-
on the translation quality. CL-SCL and our proposed method
tions simultaneously, with the assumption that word clusters
do not need any translator or parallel corpora. Actually,
can be partitioned, respectively, into common and specific
our ssSCL-ST is a combination of the extended CL-SCL
clusters. We set the numbers of common and specific clusters
and semisupervised learning technique, which learns low-
both to 10, as the authors suggested.
dimensional feature projection by using extended CL-SCL,
TriTL: Tri-TL [48], [49] divides concepts into three catego-
and then finishes space transfer by employing semisuper-
rizes, that is, identical concepts, alike concepts, and distinct
vised learning [26], [59] to select confidential target-language
concepts, and exploits both shared and distinct concepts for
examples from auxiliary unlabeled data.
cross-domain text classification. The numbers of the three
concepts are set to 5, 5, and 10, respectively, in order.
Co-Training: This state-of-the-art co-training is also based C. Evaluation Measures
on SVM [15], in which each English training example is trans-
Here, we adopt widely used precision (p), recall (r), and
lated into Chinese ones and two monolingual SVM classifiers
F-measure (F1 ) [60], [61] to measure the performance of each
are trained on them. Then, the most confidential instances are
class for classification tasks. The F1 function is obtained as
5 [Link]
6 [Link] 7 The source code of CL-SCL is available at [Link]

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

8 IEEE TRANSACTIONS ON CYBERNETICS

TABLE I
C OMPARISON OF C LASSIFICATION ACCURACY also analyze the processing of translation and discover
two interesting phenomena. One observation is that it
might result in a low coverage of the vocabularies of
target language reviews. That is, machine translation
could not guarantee to have the same expression
habits with humans. The other observation is that the
orientation of one sentence may even be changed to
nonpolarity. For example, an original English review in
iHerb is “I bought this as part of an order for someone
else, so I can not comment directly on its efficacy.
However, it works. (The corresponding Chinese trans-
lation:
),” but the
translated Chinese sentence is no longer with polarity.
follows: Hence, the translation noises will affect the performance
2pr of machine-translation-based methods.
F1 = . (3)
p+r 3) DTL and TriTL are two state-of-the-art methods based
For multiclass tasks, the global F1 usually has two forms, on non-negative matrix trifactorization. On four datasets,
that is, the macro-averaged F1 (macro-F1 ) and the micro- DTL achieves the worst performances while TriTL is
averaged F1 (micro-F1 or Accuracy) as follows: comparable to MT-SVM (CN2EN). The average accu-
K racies of them are 64.9% and 66.9%, respectively. DTL
F1 (i) 2p̄r̄ and TriTL assume that source language data and target
macro-F1 = i=1 , micro-F1 = (4)
K p̄ + r̄ language one share some identical features. However,
where F1 (i) is the F1 value of the ith class, and p̄ and r̄ are this assumption is very strict for cross-lingual sentiment
the precision and recall values averaged on all classes, respec- classification. Therefore, we can observe that these kinds
tively. In the essence, macro-F1 weights all categories equally, of methods perform worse through our experiments.
and thus can indicate poor results of minority categories. In 4) Co-training achieves the second-best performance in
contrast, micro-F1 or Accuracy assigns the same weight to all terms of averaged accuracy, but it requires parallel cor-
instances, and thus might be dominated by the performance pora both in training and predicting steps. Moreover,
of majority categories. we also observe that co-training is not stable and we
provide the experimental analysis in the following sec-
tion. Besides, it is difficult to predefine the number of
V. E XPERIMENTAL R ESULTS
iteration for avoiding negative transfer. For example,
A. Overall Performance Gui et al. [22] improved the co-training method by nega-
We compare our approach with several state-of-the-art tive transfer detection (in short, co-training with NTD),
cross-language sentiment classification methods, and we have but we find the averaged accuracy of co-training with
following observations. NTD is very close to that of co-training.
1) As shown in Table I, our ssSCL-ST consistently per- 5) The performance of neural-network-based methods are
forms better than the rest methods in terms of averaged also comparable, as shown in Table I. Specifically, the
accuracy. The averaged accuracy of ssSCL-TL reaches averaged accuracy of BSWE is 80.7% on the NLP&CC
up to 82.2%. Compared with MT-SVM, theimprovement 2013 dataset, which is close to co-training but is still
of our method is up to 13%. Moreover, ssSCL-ST is lower 1.3% than our proposed ssSCL-ST. We also
significantly better than DTL and TriTL because the observe that BSWE is not stable and could be affected
improvement of averaged accuracy is up to 15%. For dramatically by the length of reviews and the number
CL-SCL, the averaged accuracy of our proposed ssSCL- of sentiment words. For example, BSWE only achieved
ST is higher 6% than it, which demonstrates that the 53.3% accuracy on the iHerb dataset, and the reasons
semisupervised learning strategy plays an important role are that the lengths of reviews in iHerb are much shorter
in improving the prediction ability. We will discuss than those in the NLP&CC 2013 dataset and the total
the role in the following section. Co-training and its number of sentiment words in iHerb is only 1165, which
improved method with negative transfer detection per- is much fewer than that in NLP&CC 2013. Therefore,
form the second-best results, but the averaged accuracy the lack of sentiment words will lead to that BSWE
is still lower 2% than our proposed ssSCL-ST. could not learn better bilingual sentiment word embed-
2) MT-SVM methods achieve lower accuracy, that is, dings on the iHerb dataset. WD-GRL is more stable
69.4% and 67.5%, respectively. The underlying reason than BSWE method, it achieves similar performance
is that automatic machine translation performs poor compared with co-training and the averaged accuracy is
quality and causes much noise. There is an additional 79.2%. However, the averaged accuracy of our ssSCL-
aspect of this: MT-SVM can achieve higher accuracy CL is 3.0% higher than that of WD-GRL. We first
when we translate English reviews to Chinese ones. We

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

WANG et al.: CROSS-LINGUAL KNOWLEDGE TRANSFERRING 9

Fig. 4. Visualization of extended pivots in ssSCL-ST.

Fig. 3. Visualization of original pivots in CL-SCL.

B. Effectiveness of Pivot Extension

reimplemented RBST (RBST-s and RBST-hp) based on In our proposed method, we develop a one-to-many exten-
Pytorch and obtained the experimental results based sion strategy of pivot set by WordNet, in which each English
word in original pivot set is expended into a group of syn-
on the same feature space. The experimental results
onym, and we also compared our one-to-many strategy with
are shown in Table I. On dataset NLP&CC2013, com-
one-to-one strategy in CL-SCL. First, Figs. 3 and 4 show the
pared to the original models in [8], our reimplemented
visualization of pivots between one-to-one strategy in CL-SCL
RBST-s and RBST-hp have achieved 23% improvement
and our one-to-many strategy. Each dot stands for a pivot and
in terms of accuracy, which are trained and tested on the
the line between two dots the mapping between the English
same feature space with our ssSCL-TL. Since RBST
pivot and the corresponding Chinese pivot. The red dots in
divides the training set and the test set randomly, our
Fig. 4 are the extended pivots found by our one-to-many strat-
experimental results are the averaged accuracy of five
egy. We can observe that the lines between Chinese pivots and
random experiments. For the dataset iHerb proposed in
English pivots increase more because more English pivots are
our paper, the training set in iHerb is from English
added by our proposed one-to-many strategy. Second, we also
reviews, and the test set is from Chinese reviews. Due to
design experiments to compare the classification accuracy with
the different between the translated data and the original
or without pivot extension. As shown in Table II, the average
data, RBST cannot align the Chinese and English data
accuracy of (CL-SCL +WordNet) is improved over 1% when
well in the test set and achieves worse performance on
synonym expansion of WordNet is used, compared with tra-
dataset iHerb. In general, although RBST achieves bet-
ditional CL-SCL. This demonstrates that synonym expansion
ter performance on the NLP&CC13 dataset, it achieves
is helpful for pivot selection. When space transfer is consid-
worst performance on the iHerb dataset. From Table I,
ered, the average accuracy of (CL-SCL +WordNet+ST) is
we can observe that our ssSCL-TL can achieve compa-
improved about 2.6%. As shown in the last row in Table II,
rable results, but our method takes less time because we
the averaged accuracy of our proposed ssSCL-ST improves up
do not adopt the deep learning framework.
to 5% than traditional CL-SCL method.
Through the above analysis on experimental results, we
can summarize that our proposed ssSCL-ST method has two
advantages to achieve the best performance on cross-lingual
sentiment classification tasks. The first one is that ssSCL-ST C. Improvement of ssSCL-ST
employs extended CL-SCL to learn low-dimensional feature In ssSCL-ST, there are three classifiers, that is, fS (CL-
projection between the source-lingual dataset and target- SCL+WordNet), fT,L (CL-SCL+WordNet+ST), and
lingual one, which can avoid the noises of machine translation. fT (ssSCL-ST), which are trained on different extension
The second one is that ssSCL-ST sufficiently makes use strategy and training data. We show the improvement of the
of target-lingual auxiliary unlabeled data by adopting the three classifiers in Table II. We can observe that the synonym
semisupervised learning [59] strategy to select confidential expansion strategy improves the accuracy for 1% while space
examples, which helps to finish the space transfer from the transfer improves about 1.6%, and semisupervised learning
low-dimensional feature space to target-lingual feature space. improves by 2.4%. We also extend CL-SCL with space

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

10 IEEE TRANSACTIONS ON CYBERNETICS

TABLE II
ACCURACY P ERFORMANCE OF THE E XTENSION M ETHOD AND M ULTIPLE C LASSIFICATION S TRATEGIES

Fig. 5. Influence of instance selection to co-training.

transfer, denoted as CL-SCL+WordNet+ST. From the exper- the total number of selected instances from unlabeled data
iments shown in Table II, we can observe that the accuracy reaches up to 350, the performance of co-training method on
of CL-SCL+WordNet+ST reaches up to 79.6%. The result the DVD dataset begins to decrease. For the MUSIC dataset,
implies that our proposed semisupervised learning example the performance descends dramatically when the total number
selection strategy is also beneficial to CL-SCL method. For of selected instances is bigger than 150. The above observation
averaged accuracy, our proposed ssSCL-ST improves 5% over demonstrates that the overall performance of co-training is not
CL-SCL. In addition, we also conduct a series of ten trial stable and it is difficult to decide where to end the iteration
experiments to verify the significance between CL-SCL and process.
ssSCL-ST. In the resampled paired t test [62], our ssSCL-ST
achieves significantly better performance than CL-SCL for
CLSA tasks with p-value < 0.01. VI. C ONCLUSION
For cross-lingual sentiment classification tasks, we proposed
a semisupervised learning approach with SCL and space trans-
D. Stability Analysis fer, in which we proposed a one-to-many mapping strategy
In our ssSCL-ST approach, we have two instance selection to enhance the connection of pivots between two domains,
steps by semisupervised learning. The first step is to select top and then we employed a semisupervised learning paradigm to
N confidential instances from target language unlabeled data select confidential instances from unlabeled data in the tar-
to learn classifier fT,L under a low-dimensional common fea- get language domain. Compared with traditional SCL-CL, our
ture space. Next, the second step is to select 100r% instances proposed model restored the lost valuable information during
from N instances under the target language feature space. From projection by using unlabeled documents in the target lan-
experiments, we can observe that the number of instances N guage domain twice. In other words, a part of the valuable
and the ratios (r) of the training instances have minor influence information lost in the knowledge transfer process is recov-
on three datasets in terms of accuracy. However, the stability ered by transferring the feature spaces with the semisupervised
does not exist for the co-training method, which also depends learning technique. The experimental results on common prod-
on the selection of instances from unlabeled data. As shown uct review datasets validate the effectiveness of our proposed
in Fig. 5, we analyze the influence of instance selection for model. There are some directions for our future work, for
the co-training method and CT-best is the best accuracy value example, we could expand the application of our model to var-
achieved by co-training. We can observe that there exists nega- ious cross-language and cross-domain applications. Moreover,
tive transfer on DVD and MUSIC datasets. For example, when the credible knowledge between domains and the boosting

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

WANG et al.: CROSS-LINGUAL KNOWLEDGE TRANSFERRING 11

framework [21] enlighten us to further improve our space [26] Z. Yu et al., “Adaptive semi-supervised classifier ensemble for high
transferring framework in the future. dimensional data classification,” IEEE Trans. Cybern., vol. 49, no. 2,
pp. 366–379, Feb. 2019.
[27] Y. Chen, S. Song, S. Li, L. Yang, and C. Wu, “Domain space transfer
extreme learning machine for domain adaptation,” IEEE Trans. Cybern.,
R EFERENCES vol. 49, no. 5, pp. 1909–1922, May 2019.
[1] R. Mihalcea, C. Banea, and J. Wiebe, “Learning multilingual subjective [28] B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs up? Sentiment clas-
language via cross-lingual projections,” in Proc. Annu. Meeting Assoc. sification using machine learning techniques,” in Proc. Annu. Meeting
Comput. Linguist. (ACL), Jun. 2007, pp. 976–983. Assoc. Comput. Linguist. (ACL), 2002, pp. 79–86.
[2] E. Asgarian, M. Kahani, and S. Sharifi, “The impact of sentiment fea- [29] P. D. Turney, “Thumbs up or thumbs down? Semantic orientation applied
tures on the sentiment polarity classification in persian reviews,” Cogn. to unsupervised classification of reviews,” in Proc. Annu. Meeting Assoc.
Comput., vol. 10, no. 1, pp. 117–135, 2018. Comput. Linguist. (ACL), 2002, pp. 417–424.
[3] L. Qiu, Q. Lei, and Z. Zhang, “Advanced sentiment classification of [30] G. Qiu, B. Liu, J. Bu, and C. Chen, “Expanding domain sentiment lexi-
tibetan microblogs on smart campuses based on multi-feature fusion,” con through double propagation,” in Proc. IJCAI, 2009, pp. 1199–1204.
IEEE Access, vol. 6, pp. 17896–17904, 2018. [31] T. Mullen and N. Collier, “Sentiment analysis using support vector
[4] M. Chen, W. Chen, and L. Ku, “Application of sentiment analysis to machines with diverse information sources,” in Proc. Conf. Empirical
language learning,” IEEE Access, vol. 6, pp. 24433–24442, 2018. Methods Nat. Lang. Process. (EMNLP), Jul. 2004, pp. 412–418.
[5] X. Wan, “Using bilingual knowledge and ensemble techniques for unsu- [32] L. Zhao, M. Huang, J. Sun, H. Luo, X. Yang, and X. Zhu, “Sentiment
pervised chinese sentiment analysis,” in Proc. Conf. Empirical Methods extraction by leveraging aspect-opinion association structure,” in Proc.
Nat. Lang. Process. (EMNLP), 2008, pp. 553–561. Int. Conf. Inf. Knowl. Manage. (CIKM), 2015, pp. 343–352.
[6] P. Prettenhofer and B. Stein, “Cross-language text classification using [33] D. Davidov, O. Tsur, and A. Rappoport, “Enhanced sentiment learning
structural correspondence learning,” in Proc. Annu. Meeting Assoc. using twitter hashtags and smileys,” in Proc. ICCL, 2010, pp. 241–249.
Comput. Linguist. (ACL), 2010, pp. 1118–1127. [34] R. Xia, C. Wang, X. Dai, and T. Li, “Co-training for semi-supervised
sentiment classification based on dual-view bags-of-words representa-
[7] N. Li, S. Zhai, Z. Zhang, and B. Liu, “Structural correspondence learning
tion,” in Proc. Annu. Meeting Assoc. Comput. Linguist. (ACL), 2015,
for cross-lingual sentiment classification with one-to-many mappings,”
pp. 1054–1063.
in Proc. AAAI Conf. Artif. Intell. (AAAI), 2017, pp. 3490–3496.
[35] C. Ma, M. Wang, and X. Chen, “Topic and sentiment unification maxi-
[8] Q. Chen, C. Li, and W. Li, “Modeling language discrepancy for cross-
mum entropy model for online review analysis,” in Proc. WWW, 2015,
lingual sentiment analysis,” in Proc. Int. Conf. Inf. Knowl. Manage.
pp. 649–654.
(CIKM), 2017, pp. 117–126.
[36] G. Wang, J. Lu, K. Choi, and G. Zhang, “A transfer-based additive LS-
[9] J. Li, K. Lu, Z. Huang, L. Zhu, and H. T. Shen, “Transfer independently
SVM classifier for handling missing data,” IEEE Trans. Cybern., vol. 50,
together: A generalized framework for domain adaptation,” IEEE Trans.
no. 2, pp. 739–752, Feb. 2020.
Cybern., vol. 49, no. 6, pp. 2144–2155, Jun. 2019.
[37] J. Li, S. Qiu, Y. Shen, C. Liu, and H. He, “Multisource transfer learn-
[10] Z. Lin et al., “An unsupervised cross-lingual topic model framework for
ing for cross-subject EEG emotion recognition,” IEEE Trans. Cybern.,
sentiment classification,” IEEE/ACM Trans. Audio, Speech, Language
vol. 50, no. 7, pp. 3281–3293, Jul. 2020.
Process., vol. 24, no. 3, pp. 432–444, Mar. 2016.
[38] C. Banea, R. Mihalcea, J. Wiebe, and S. Hassan, “Multilingual sub-
[11] X. Zhou, X. Wan, and J. Xiao, “Cross-lingual sentiment classifica- jectivity analysis using machine translation,” in Proc. Conf. Empirical
tion with bilingual document representation learning,” in Proc. Annu. Methods Nat. Lang. Process. (EMNLP), 2008, pp. 127–135.
Meeting Assoc. Comput. Linguist. (ACL), 2016, pp. 1403–1412. [39] C. Banea, R. Mihalcea, and J. Wiebe, “Multilingual subjectivity: Are
[12] C. Banea, R. Mihalcea, and J. Wiebe, “A bootstrapping method for more languages better?” in Proc. ICCL, 2010, pp. 28–36.
building subjectivity lexicons for languages with scarce resources,” in [40] M. Hajmohammadi, R. Ibrahim, A. Selamat, and H. Fujita,
Proc. LREC, 2008, pp. 1–4. “Combination of active learning and self-training for cross-lingual sen-
[13] I. A. Sag, T. Baldwin, F. Bond, A. Copestake, and D. Flickinger, timent classification with density analysis of unlabelled samples,” Inf.
“Multiword expressions: A pain in the neck for NLP,” in Proc. CICLing, Sci., vol. 317, pp. 67–77, Oct. 2015.
2002, pp. 1–15. [41] M. Hajmohammadi, R. Ibrahim, and A. Selamat, “Cross-lingual sen-
[14] D. Anastasiou, “Idiom Treatment Experiments in Machine Translation.” timent classification using multiple source languages in multi-view
Newcastle upon Tyne, U.K.: Cambridge Scholars, 2010, pp. 998–1004. semi-supervised learning,” Eng. Appl. Artif. Intell., vol. 36, pp. 195–203,
[15] X. Wan, “Co-training for cross-lingual sentiment classification,” in Proc. Nov. 2014.
Annu. Meeting Assoc. Comput. Linguist. (ACL)’09, 2009, pp. 235–243. [42] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” J.
[16] X. Meng, F. Wei, X. Liu, M. Zhou, G. Xu, and H. Wang, “Cross-lingual Mach. Learn. Res., vol. 3, pp. 993–1022, Jan. 2003.
mixture model for sentiment classification,” in Proc. Annu. Meeting [43] G. Zhou, T. He, J. Zhao, and W. Wu, “A subspace learning framework
Assoc. Comput. Linguist. (ACL)’12, 2012, pp. 572–581. for cross-lingual sentiment classification with partial parallel data,” in
[17] M. Paul and R. Girju, “Cross-cultural analysis of blogs and forums with Proc. IJCAI, 2015, pp. 1426–1433.
mixed-collection topic models,” in Proc. Conf. Empirical Methods Nat. [44] B. Jing, C. Lu, D. Wang, F. Zhuang, and C. Niu, “Cross-domain labeled
Lang. Process. (EMNLP), 2009, pp. 1408–1417. lda for cross-domain text classification,” in Proc. IEEE Int. Conf. Data
[18] L. Li, X. Jin, and M. Long, “Topic correlation analysis for cross-domain Min. (ICDM), 2018, pp. 187–196.
text classification,” in Proc. AAAI Conf. Artif. Intell. (AAAI), 2012. [45] N. Zainuddin, A. Selamat, and R. Ibrahim, “Hybrid sentiment classifi-
[19] Y. Bao, N. Collier, and A. Datta, “A partially supervised cross-collection cation on twitter aspect-based sentiment analysis,” Appl. Intell., vol. 48,
topic model for cross-domain text classification,” in Proc. Int. Conf. Inf. pp. 1218–1232, May 2018.
Knowl. Manage. (CIKM), 2013, pp. 239–248. [46] Y. Wang, M. Huang, X. Zhu, and L. Zhao, “Attention-based LSTM for
[20] Z. Lin, X. Jin, X. Xu, W. Wang, X. Cheng, and Y. Wang, “A cross- aspect-level sentiment classification,” in Proc. Conf. Empirical Methods
lingual joint aspect/sentiment model for sentiment analysis,” in Proc. Nat. Lang. Process. (EMNLP), 2016, pp. 606–615.
Int. Conf. Inf. Knowl. Manage. (CIKM), 2014, pp. 1089–1098. [47] F. Zhuang, P. Luo, P. Yin, Q. He, and Z. Shi, “Concept learning for
[21] Q. Chen, W. Li, Y. Lei, X. Liu, and Y. He, “Learning to adapt credible cross-domain text classification: A general probabilistic framework,” in
knowledge in cross-lingual sentiment analysis,” in Proc. 53rd ACL, 2015, Proc. IJCAI, 2013, pp. 1960–1966.
pp. 419–429. [48] F. Zhuang, P. Luo, C. Du, Q. He, and Z. Shi, “Triplex transfer learning:
[22] L. Gui et al., “Cross-lingual opinion analysis via negative transfer detec- Exploiting both shared and distinct concepts for text classification,” in
tion,” in Proc. Annu. Meeting Assoc. Comput. Linguist. (ACL), Jun. 2014, Proc. WSDM, 2013, pp. 425–434.
pp. 860–865. [49] F. Zhuang, P. Luo, C. Du, Q. He, Z. Shi, and H. Xiong, “Triplex
[23] P. Prettenhofer and B. Stein, “Cross-lingual adaptation using structural transfer learning: Exploiting both shared and distinct concepts for text
correspondence learning,” ACM Trans. Intell. Syst. Technol., vol. 3, no. 1, classification,” IEEE Trans. Cybern., vol. 44, no. 7, pp. 1191–1203,
pp. 1–22, 2011. Jul. 2014.
[24] X. He, H. Zhang, W. Chao, and D. Wang, “Semi-supervised learning [50] X. Hu, J. Pan, P. Li, H. Li, W. He, and Y. Zhang, “Multi-bridge transfer
on cross-lingual sentiment analysis with space transfer,” in Proc. IEEE learning,” Knowl. Based Syst., vol. 97, pp. 60–74, Apr. 2016.
BigDataService, 2015, pp. 371–377. [51] J. Blitzer, R. McDonald, and F. Pereira, “Domain adaptation with struc-
[25] M. Long, J. Wang, G. Ding, W. Cheng, X. Zhang, and W. Wang, “Dual tural correspondence learning,” in Proc. Conf. Empirical Methods Nat.
transfer learning,” in Proc. SDM, 2012, pp. 540–551. Lang. Process. (EMNLP), 2006, pp. 120–128.

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

12 IEEE TRANSACTIONS ON CYBERNETICS

[52] J. Blitzer, M. Dredze, and F. Pereira, “Biographies, bollywood, boom- Jingyuan Yang received the Ph.D. degree in information technology from
boxes and blenders: Domain adaptation for sentiment classification,” Rutgers Business School at Rutgers, State University of New Jersey, New
in Proc. Annu. Meeting Assoc. Comput. Linguist. (ACL), Jun. 2007, Brunswick, NJ, USA, in 2018.
pp. 440–447. She is currently an Assistant Professor in information systems and opera-
[53] R. K. Andoa and T. Zhang, “A framework for learning predictive struc- tions management area with the School of Business, George Mason University,
tures from multiple tasks and unlabeled data,” J. Mach. Learn. Res., Fairfax, VA, USA. Her primary research interests include data mining, big
vol. 6, pp. 1817–1853, Nov. 2005. data, machine learning, and business analytics.
[54] R. K. Ando and T. Zhang, “A high-performance semi-supervised learn-
ing method for text chunking,” in Proc. Annu. Meeting Assoc. Comput.
Linguist. (ACL), 2005, pp. 1–9.
[55] T. Zhang, “Solving large scale linear prediction problems using stochas-
tic gradient descent algorithms,” in Proc. ICML, 2004, pp. 116–125.
[56] J. Duchi, S. Shalev-Shwartz, Y. Singer, and T. Chandra, “Efficient pro-
jections onto the L1-ball for learning in high dimensions,” in Proc.
Baoyu Jing received the master’s degree from Carnegie Mellon University,
ICML, 2008, pp. 272–279.
Pittsburgh, PA, USA, in 2018. He is currently pursuing the Ph.D. degree
[57] H. Zhou, L. Chen, F. Shi, and D. Huang, “Learning bilingual sentiment
with the Department of Computer Science, University of Illinois at Urbana–
word embeddings for cross-language sentiment classification,” in Proc.
Champaign, Champaign, IL, USA.
Annu. Meeting Assoc. Comput. Linguist. (ACL), 2015, pp. 430–440.
His research interests include transfer learning and topic modeling.
[58] J. Shen, Y. Qu, W. Zhang, and Y. Yu, “Wasserstein distance guided rep-
resentation learning for domain adaptation,” in Proc. AAAI Conf. Artif.
Intell. (AAAI), 2018, pp. 4058–4065.
[59] Z. Liu, X. Dong, Y. Guan, and J. Yang, “Reserved self-training: A semi-
supervised sentiment classification method for chinese microblogs.” in
Proc. IJCNLP, 2013, pp. 455–462.
[60] D. D. Lewis, “Evaluating and optimizing autonomous text classification
systems,” in Proc. SIGIR, 1995, pp. 246–254. Wenjie Zhang is currently pursuing the master’s degree in computer science
[61] F. Sebastiani, “Machine learning in automated text categorization,” ACM from Beihang University, Beijing, China.
Comput. Surveys, vol. 34, no. 1, pp. 1–47, 2002. His main research interests include transfer learning and sentiment analysis.
[62] J. Demšar, “Statistical comparisons of classifiers over multiple data sets,”
J. Mach. Learn. Res., vol. 7, pp. 1–30, Dec. 2006.

Deqing Wang (Member, IEEE) received the Doctoral degree in computer Xiaonan He received the master’s degree in computer science from Beihang
science from Beihang University, Beijing, China, in 2013. University, Beijing, China, in 2015.
He is an Assistant Professor with the School of Computer Science, Beihang He is currently an Engineering with Baidu, Inc., Beijing. His main research
University. His research focuses on text categorization and data mining for interests include sentiment analysis and information retrieval.
software engineering and machine learning.

Junjie Wu received the Ph.D. degree in management science and engineering

from Tsinghua University, Beijing, China, in 2008.
He is a Professor with the School of Economics and Management, Beihang Hui Zhang received the M.S. and Ph.D. degrees in computer science from
University, Beijing. Beihang University, Beijing, China, in 1994 and 2009, respectively.
Prof. Wu is a recipient of various nationwide academic awards, including He is currently a Professor with the School of Computer Science and
the NSFC Distinguished Young Scholars and the MOE Changjiang Young Engineering, Beihang University. His main research interests include cloud
Scholars. computing, web information retrieval, and data mining.

Authorized licensed use limited to: University Visveswarayya College of Engineering. Downloaded on March 04,2021 at 18:59:02 UTC from IEEE Xplore. Restrictions apply.