Expert Systems With Applications 199 (2022) 116886

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: [Link]/locate/eswa

Stock Ranking with Multi-Task Learning

Tao Ma, Ying Tan ∗
Key Laboratory of Machine Perceptron (MOE), School of Artificial Intelligence, Institute for Artificial Intelligence, Peking University, Beijing 100871, China

ARTICLE INFO ABSTRACT

Keywords: Stock prediction, aiming at predicting the future trends of stocks, plays a key role in stock investment. Towards
Stock prediction the investment target, the primary task is selecting the stocks with potentials to obtain the highest excess
Stock ranking returns, always regarded as stock ranking. List-wise stock ranking is able to consider the relative comparisons
Multi-task learning
of multiple stocks, approaching the essence of stock ranking most. However, most existing methods fail in
Deep neural networks
list-wise stock ranking, because the information complexity and small number of samples bring in training
difficulties.
To address these limitations, a novel Deep Multi-Task Learning (DMTL) solution is proposed, called Multi-
Task Stock Ranking (MTSR). It utilizes the joint learning framework of DMTL to learn the list-wise stock
ranking with the enhancements of auxiliary tasks. With DMTL, the easily-trained tasks act as learning guider,
providing extra gradient backpropagation, to help learn the hardly-trained list-wise ranking task. Additionally,
Task Relation Attention is utilized to capture the dynamic task relations to achieve better knowledge transfer
between tasks. The experiments conducted on real-world stock datasets demonstrate the superiority of MTSR
over several state-of-the-art methods.

1. Introduction Auto-regressive Model (VAR) (Johansen, 1991). However, there are

three primary drawbacks in these methods: (1) These methods strongly
Stock prediction, aiming to predict the future trends of stocks, rely on the feature (e.g. stock prices or technical indicators) selections,
plays a key role in stock investments (Preethi & Santhi, 2012). If which are usually manually constructed; (2) These methods rely on the
the trading strategies are based on activate portfolios, the expected hypothesized stochastic processes, which hardly capture the complex
profits strongly depend on whether the stocks with the best profitability sequential patterns in stock data and handle the volatility in stock
can be selected. Towards this investment target, the problem of stock
markets when dealing with complex time series prediction tasks; (3)
selection is always regarded as stock ranking. The methods of stock
Most of these traditional methods pay more attention to the variable
ranking can be categorized into: point-wise, pair-wise, and list-wise
interpretation rather than the improved prediction performance. To ad-
ranking predictions, according to the number of stocks processed in
the prediction process. List-wise stock ranking is able to consider the dress the shortages of traditional methods, there have been many efforts
relative comparisons of multiple stocks, approaching the essence of bringing in machine learning methods to model the volatility and non-
stock ranking most. However, it brings in a lot of training difficulties linear structure of stock data, such as Neural Networks (Chakraborty,
for the learning-based methods because the information complexity and Mehrotra, Mohan, & Ranka, 1992; Zhang, 2003) and Support Vector
small number of samples. Deep Multi-Task Learning (DMTL) is able to Machine (Gu, Tan, & He, 2010; Kim, 2003; Pai & Lin, 2005; Tan,
utilize the auxiliary tasks to benefit the learning process of the primary Xia, & Wang, 2000). Although they have made improvements over
task (i.e., the target task). Therefore, this paper aims to utilize the joint traditional methods, they still perform poorly when facing with the
learning framework of DMTL to address the training difficulty of list- complex prediction problem in stock markets, especially processing a
wise stock ranking, enhancing the stability of the training process and large number of stocks.
improving the stock prediction performance. Deep learning is regarded as a promising solution for stock pre-
There is a rich history of the researches about stock prediction diction, because of its strong capability of discovering the complex
in the traditional finance industry. Most of the traditional solutions
patterns from a large amount of data. A few attempts with deep neural
for stock prediction are based upon the time series analysis models,
networks have been made for stock prediction (Deng et al., 2019; Feng
such as Moving Average (MA) (Said & Dickey, 1984), Auto-regressive
et al., 2019b; Fischer & Krauss, 2018; Li, Song, & Tao, 2019; Wang,
Integrated Moving Average (ARMA) (Box & Pierce, 1970), and Vector

∗ Corresponding author.
E-mail addresses: pku_mark@[Link] (T. Ma), ytan@[Link] (Y. Tan).

[Link]
Received 29 August 2021; Received in revised form 10 March 2022; Accepted 11 March 2022
Available online 23 March 2022
0957-4174/© 2022 Elsevier Ltd. All rights reserved.
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Zhang, Tang, Wu, & Xiong, 2019). To evaluate the performance of the 2. Related works
proposed method, this paper selects several state-of-the-art solutions
serving as baselines. Zhang, Aggarwal, and Qi (2017) proposed a point- 2.1. Stock prediction
wise stock prediction method, called State Frequency Memory (SFM),
which aimed to discover multi-frequency trading patterns in stock Stock prediction, aims to predict the future trends of stocks, having
time series. Zhang and Tan (2018) used LSTM-RNN networks for list- a long history in the field of economics. Most of the widely-used tradi-
wise stock ranking, called Deep Stock Ranker (DSR). Ma and Tan tional methods are based upon the time series analysis, such as Moving
(2020c) used DMTL to utilize the stock correlations for point-wise stock Average (MA) (Said & Dickey, 1984), Auto-regressive Integrated Mov-
prediction, called Multiple Stock Jointly Forecasting (MSJF). Although ing Average (ARMA) (Box & Pierce, 1970), and Vector Auto-regressive
there have been some promising results in these methods, there are still Model (VAR) (Johansen, 1991). However, most of these traditional
several drawbacks: (1) For the solutions for point-wise predictions, such methods pay more attention to the variable interpretation rather than
as SFM, they predict the stock trends separately, only considering the the improved prediction performance. Especially when dealing with
inner patterns in individual stocks, ignoring the relative comparisons complex time series prediction tasks, they yield apparent drawbacks.
between different stocks; (2) For the solutions proposed for list-wise To address the shortages of traditional methods, there have been many
ranking, such as DSR, they do not well address the training difficulty efforts bringing in machine learning methods to model the volatility
of list-wise ranking task; (3) For the solutions with DMTL, such as and non-linear structure of stock data (Pai & Lin, 2005; Zhang, 2003).
MSJF, they use the MTL framework to process the correlations between However, they still perform poorly when facing the problem dealing
different stocks, not use it to perform list-wise ranking task and address with a large amount of stock data.
its training difficulty, different from ours. Deep learning is strongly capable of extracting the complex latent
To address the limitations in the existing methods, this paper pro- patterns form a large amount of data, which has been widely used
posed a DMTL based solution for list-wise stock ranking, called Multi- in several applications (Hashimoto, Xiong, Tsuruoka, & Socher, 2016;
Task Stock Ranking (MTSR). In the proposed method, the target task is Lu et al., 2017; Tan, Deng, & Ruan, 2009). Therefore, it has been
assigned as list-wise stock ranking task, i.e., the primary task in DMTL, recently regarded as a promising solution for complex stock prediction
which aims to directly predict the future ranking results for a list of problems. For some concrete examples, Fischer and Krauss (2018)
stocks in terms of a desired criteria, such as expected excess returns. To applied long-short term memory RNN (LSTM-RNN) for stock market
address the training difficulty of list-wise ranking task, MTSR utilizes prediction; Zhang et al. (2017) guided the learning process for stock
the joint learning framework of DMTL. Specifically, several auxiliary prediction by discovering multi-frequency trading patterns; Zhang and
tasks, which are relatively easier to be trained, act as learning guider Tan (2018) proposed a list-wise ranking method for stock prediction
to guide the training process of hardly-trained list-wise ranking task, using LSTM-RNN; Deng et al. (2019) employed dilated temporal con-
volutional neural networks to predict the future price trends of stocks;
providing extra gradient backpropagation to help train the networks.
and Wang et al. (2019) proposed a buying-winners-and-selling-losers
Additionally, to achieve better knowledge transfer between different
investment strategy using deep reinforcement attention networks. Ma
tasks, Task Relation Attention Networks (TRAN) (Ma & Tan, 2020a) is
and Tan (2020c) proposed a Multi-Task Learning (MTL) framework
utilized to capture the dynamic task correlations to enhance the feature
to predict the future trends of multiple stocks simultaneously. Xu,
learning process. The overall framework of MTSR is illustrated in
Zhang, Ye, Zhao, and Tan (2020) used an attention module with
Fig. 4, which contains three primary modules: shared encoding layers,
fixed temporal windows to capture the local sequential patterns for
task-specific encoding layers with TRAN, and task-specific prediction
stock prediction. Ding, Wu, Sun, Guo, and Guo (2020) enhanced the
layers.
temporal locality in basic Transformer with the mechanism of Gaussian
To evaluate the performances of MTSR, the experiments are con-
Priors and fixed temporal windows to capture the multi-scale temporal
structed on the datasets from real-world stock markets, including both
patterns in stock time series.
Chinese and American markets. Several state-of-the-art stock predic-
Although these solutions have shown promising results for stock
tion methods, including SFM, DSR and MSJF, serve as baselines for
prediction, there are still a few drawbacks: (1) Most of the exist-
comparisons. The experimental results demonstrate the effectiveness
ing methods preferred the easily-trained point-wise prediction tasks,
of MTSR and its superiority over other baseline methods. The primary
e.g., the point-wise classification for the future trends (raise or fall) of
contributions of this paper can be summarized as follows: stock prices, instead of list-wise ranking task, which is more difficult
• This paper proposes a novel solution based on Deep Multi-Task to be trained but approaches the investment target most; (2) For some
Learning for list-wise stock ranking prediction, called Multi-Task solutions of list-wise ranking, they did not well address the training
Stock Ranking (MTSR); difficulty of the ranking task for a large number of stocks.
• Within the joint learning framework of DMTL, the proposed
method uses several easily-trained auxiliary tasks acting as learn- 2.2. Multi-task learning
ing guider to help address the training difficulty of hardly-trained
list-wise ranking task; Multi-task Learning (MTL) provides an efficient framework for
• To our best knowledge, this is the first time to utilize DMTL to leveraging task relationships to achieve knowledge transfer among
solve the list-wise stock ranking problem, which aims to address tasks so that it can lead to improved performance. MTL with deep
its training difficulty in the existing methods; neural networks has been used widely and successfully across many
• The experiments conducted on real-world stock datasets demon- applications of machine learning, from natural language processing
strate the effectiveness of MTSR, and its superiority over several (Hashimoto et al., 2016; Luong, Le, Sutskever, Vinyals, & Kaiser, 2015),
state-of-the-art solutions. speech recognition (Deng, Hinton, & Kingsbury, 2013), to computer
vision (Abdulnabi, Wang, Lu, & Jia, 2015; Girshick, 2015; Lu et al.,
The reminder of this paper is organized as follows: Section 2 in- 2017).
troduces the preliminary knowledge about stock ranking, multi-task The typical MTL architectures were sharing the bottom layers for
learning, TRAN and LSTM networks; Section 3 introduces the details all tasks and split top layers for different tasks, proposed by Caruana
of our proposed method MTSR; Section 4 introduces the constructed (1997). Afterwards, there have been some attempts to design partly
experiments, the performance comparison, the back-testing results and shared architectures between tasks, instead of only sharing the bot-
the analyses. Section 5 is the conclusion of this paper. tom hidden layers. For some recent example, Duong, Cohn, Bird, and

2
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Cook (2015) performed regularization constraints parameters sharing

between sources and target languages.
For leveraging the task relationships in MTL, there are some re-
cent examples. The cross-stitch networks (Misra, Shrivastava, Gupta,
& Hebert, 2016) learned an optimal combination of task-specific rep-
resentations for each task using linear units. The tensor factorization Fig. 1. Comparison between three methods of stock prediction.
model (Yang & Hospedales, 2016) generated the hidden layer parame-
ters for each task. The multi-gate mixture-of-experts model (Ma et al.,
2018) using gating mechanism to capture the task differences and 3.1.2. Pair-wise prediction
implicitly model the task relationships. In the works of Zhao et al. Pair-wise prediction is aimed to give the comparison results for
(2019), they applied attention networks to statically capture the task arbitrary two individual stocks, determining which stock is better in
relationships. Ma and Tan (2020b) proposed an attention based model the given two. After the pair-wise comparison of all stocks, the final
to achieve adaptive and dynamic knowledge transfer between different ranking results are obtained, the process of which is similar to Bubble
tasks according to task relationships. Sorting.
(𝑠̂𝑖 , 𝑠̂𝑗 ) = 𝑓𝑝 (𝐱𝑖 , 𝐱𝑗 ), 𝑖, 𝑗 = 1, 2, … , 𝐾,
[ ] (3)
3. Preliminaries 𝐬̂ = 𝑠̂1 , 𝑠̂2 , … , 𝑠̂𝐾 ,
where 𝑓𝑝 (⋅) represents the pair-wise prediction function for comparing
In this section, the basic definitions and background contents about each pair of stocks.
the proposed method are presented. To more clearly describe the Different from the way of point-wise prediction, pair-wise prediction
related definitions and formulas, this paper uses: is able to consider the relative comparison between stocks. However,
there is the requirement of comparing every single stock with the others
• bold capital letters (e.g., 𝐗) to represent matrices; in pair-wise, which requires the complexity of 𝑂(𝑛2 ). Some recent works
• bold lowercase letters (e.g., 𝐱) to represent vectors; have been proposed, such as Gorynin, Monfrini, and Pieczynski (2017).
• normal lowercase letters (e.g., 𝑥) to represent scalars;
• normal lowercase Greek letters (e.g., 𝜆) to represent coefficients; 3.1.3. List-wise prediction
• 𝑋𝑖𝑗 to represent the entry at the 𝑖th row and 𝑗th column of matrix List-wise prediction is aimed to directly give the ranking predic-
𝐗. tions of multiple stocks, similar to the idea in the typical ranking
algorithms (Yu et al., 2014).
[ ] ([ ])
3.1. Stock ranking 𝐬̂ = 𝑠̂1 , 𝑠̂2 , … , 𝑠̂𝐾 = 𝑓𝑙 𝐱1 , 𝐱2 , … , 𝐱𝐾 , (4)

where 𝑓𝑙 (⋅) represents the list-wise ranking prediction function.

Stock ranking algorithm aims to give a ranking prediction to demon- Different from the approaches of point-wise prediction and pair-
strate the relative levels of investment revenues of stocks based on wise prediction, list-wise prediction is able to consider the relative
the various stock features. The general definition can be described as comparisons of all stocks in overall, approaching the essence of stock
follows: ranking. However, in the way of list-wise prediction, there are more
difficulties in calculating the loss function and guiding the training
𝒔̂ = 𝑓 (𝑿), (1) process, compared with point-wise and pair-wise predictions. There are
some recent list-wise approaches proposed for stock predictions, such
where 𝐬 is the predicted ranking values of multiple stocks and 𝑿 is the as the works of Zhang and Tan (2018) and Feng et al. (2019a).
input features of stocks.
According the amount of the stocks which are compared in the 3.1.4. Comparison between three prediction approaches
prediction process, the stock ranking algorithm can be categorized into: With the comparison between three methods (see Fig. 1), list-
point-wise prediction, pair-wise prediction and list-wise prediction. wise prediction is actually more close to the essence of stock ranking,
because it considers all the stocks at the same time and directly give
their ranking results, while point-wise prediction is processing stocks
3.1.1. Point-wise prediction separately. However, for the complexity of model learning, the model
Point-wise stock prediction is usually aimed to predict the future with point-wise prediction is much easier to be learned than the one
values (i.e., the classification of future price movements, or the regres- with list-wise prediction. For pair-wise prediction, its performance of
sion of future price values) of individual stocks independently. And stock ranking and complexity of model learning are in the middle level
then, integrating all the predicted results of individual stocks to obtain between point-wise prediction and list-wise prediction.
the final ranking results. Assuming that there are 𝐾 stocks in total, and
the procedures of list-wise prediction can be described as: 3.2. Multi-task learning

𝑠̂𝑘 = 𝑓𝑜𝑘 (𝐱𝑘 ), 𝑘 = 1, 2, … , 𝐾, Multi-task learning (MTL) aims to generalize better on the original
[ ] (2)
𝐬̂ = 𝑠̂1 , 𝑠̂2 , … , 𝑠̂𝐾 , tasks, by leveraging the shared knowledge among tasks (Ruder, 2017).
It provides an efficient framework for leveraging task relationships to
where 𝑓𝑜𝑘 (⋅) represents the point-wise prediction function for stock 𝑘.
achieve knowledge transfer among tasks so that it can lead to improved
Point-wise prediction is the typical solution for stock prediction. performance.
The key component in this solution is to handle each stock separately. The typical MTL architectures were sharing the bottom layers for
Despite the advantage of easily building the model and guiding the all tasks and split top layers for different tasks, proposed by Caruana
training process, it critically ignores the relative comparison between (1997). It is usually named as Hard Parameter Sharing in MTL, and the
stocks. In the recent years, there are several approaches about point- most commonly used approach in MTL, as presented in Fig. 2.
wise stock predictions proposed, such as the works of Li, Shen, and Afterwards, there have been some attempts to design partly shared
Zhu (2018), Ma and Tan (2020c) and Li et al. (2020). architectures between tasks, instead of only sharing the bottom hidden

3
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

feature representations; (2) the incorporated information of positively

related tasks acts as regularizer to avoid the risks of over-fitting.
The knowledge transformer mainly depends on the task relationships.
Therefore, how to appropriately model task relationships and how to
control the knowledge transfer among tasks are crucial in MTL.
To achieve better performances in MTL, the works of Ma and Tan
(2020a) called Task Relation Attention Networks (TRAN) are applied
in MTSR, to adaptively capture the task relationships and dynamically
control the knowledge transfer. Specifically, TRAN is an attention-
Fig. 2. MTL of hard parameter sharing (sharing the bottom layers). based model to adaptively capture the task relationships via task cor-
relation matrix according to their inputs and specify the shared feature
representations for different tasks. Since the task relationships are
adaptively learned by TRAN during the learning process, it is replacing
the predefined relationships.
Assuming that there are 𝑀 tasks in MTL, and their encoded features
[ ]𝑇
are 𝐏 = 𝐩1 , 𝐩2 , … , 𝐩𝑀 ∈ R𝑀×𝐹 , where 𝐹 is the output dimension of
task encoders. And the formulary definition of TRAN is as follows:
[ ]𝑇
𝐏′ = TRAN(𝐏) = 𝐀1 𝐏, 𝐀2 𝐏, … , 𝐀𝑀 𝐏 ,
( )
𝐀𝑖 = 𝛼𝑖1 , 𝛼𝑖2 , … , 𝛼𝑖𝑀 , 𝑖 = 1, 2, … , 𝑀,
( )
exp 𝑎𝑡 (𝐩𝑖 ∥ 𝐩𝑗 ) (5)
𝛼𝑖𝑗 = ∑𝑀 ( ) , 𝑗 = 1, … , 𝑀,
𝑚=1 exp 𝑎 (𝐩
𝑡 𝑖 ∥ 𝐩𝑚 )
( )
𝑎𝑡 (𝐩𝑖 ∥ 𝐩𝑗 ) = 𝑡𝑎𝑛ℎ 𝐖𝑎 (𝐩𝑖 ∥ 𝐩𝑗 ) + 𝐛𝑎 , 𝑖, 𝑗 = 1, … , 𝑀,
Fig. 3. MTL of soft parameter sharing (private models with layer constraints).
where

• 𝐏′ is the outputs of TRAN;

layers. Another commonly used MTL approach is called Soft Parameter • 𝐀𝑖 is the attention weights between other tasks and target task 𝑖;
Sharing, in which each task has its private model with independent
• 𝛼𝑖𝑗 is the attention coefficients between target task 𝑖 and other
parameters. The distance between the parameters of different models is tasks 𝑗;
then regularized in order to encourage their parameters to be similar, • ∥ is the concatenation operation;
as presented in Fig. 3, such as the works of Duong et al. (2015) per-
• 𝑎𝑡 (⋅) is the shared attention networks, implemented with a single-
forming regularization constraints parameters sharing between sources layer fully-connected neural network with activation of 𝑡𝑎𝑛ℎ.
and target languages.
In general, the improvements which MTL brings to the original The learning process of TRAN can be summarized as: (1) learn the
models can be explained from two aspects: (1) Enhancing the fea- correlations between all tasks, obtaining the attention weights; (2) use
ture learning with knowledge transfer. In the framework of MTL, the learnt attention weights to combine all the task-specific features for
multiple tasks are able to share their task-domain knowledge with each different target task to obtain their new forms of task representations.
other. The process of knowledge transfer between tasks leads the latent
representations to the abundantly informative ones. (2) Performing 3.4. Long-short term memory recurrent neural networks
regularization on the primary tasks. Specifically, the auxiliary tasks
help the primary tasks focus on the features that really matter in the Long-Short Term Memory (LSTM) is one kind of the Recurrent Neu-
problem and drop the noisy of irrelevant features, and act as regularizer ral Networks (RNNs) structures, which has been widely and successfully
by introducing an inductive bias to reduce the risk of overfitting as well applied in several applications with sequential data, such as natural
as the Rademacher complexity of the model. language processing, speech recognition or voice processing. LSTM is
In the problem of stock ranking, MTL acts as learning guider. The aimed to solve the problem of long-term dependency, which is able
primary task in our method is list-wise stock ranking prediction. How- to evolve hidden states to capture the long-term sequential patterns in
ever, there are more difficulties in the training process of this method, the input data. In specific, compared with the vanilla RNNs, LSTM is
compared with the other two. In specific, the dataset is split into equipped with the cell states to reserve the long-term information to
samples in the way of list-wise, the amount of samples is meant to be discover the long-term dependency in the input sequences, which is
able to avoid gradient vanishing in the training with Back-Propagation
small. One sample is the list containing all the stock in a single trading
Through Time (BPTT).
day, therefore, the amount of samples will be equal to the number of
Here are the formulary definitions of LSTM. Assuming that there is a
trading days. The small amount of samples will bring lots of difficulties
input sequence with the length of 𝑇 , 𝐗 = [𝐱1 , 𝐱2 , … , 𝐱𝑡 , … , 𝐱𝑇 ]𝑇 ∈ R𝑇 ×𝐷 ,
to the model training. However, on the contrary, the way of point-
where 𝐷 is the input dimension and 𝑡 represents the time step in the se-
wise prediction will split the dataset into a large amount of samples,
quence, and the output of hidden states is 𝐇 = [𝐡1 , 𝐡2 , … , 𝐡𝑡 , … , 𝐡𝑇 ]𝑇 ∈
in which the amount will be equal to the product of the number of
R𝑇 ×𝑈 , where 𝑈 is the number for hidden units.
stocks and the number of trading days. Therefore, the task of point-wise ( )
prediction is much easier to be learnt. As a result, in the framework [𝐡1 , … , 𝐡𝑡 , … , 𝐡𝑇 ]𝑇 = LSTM [𝐱1 , … , 𝐱𝑡 , … , 𝐱𝑇 ]𝑇
of MTL, we can supply several auxiliary tasks (point-wise prediction), 𝐳𝑡 = 𝑡𝑎𝑛ℎ(𝐖𝑧 𝐱𝑡 + 𝐐𝑧 𝐡𝑡−1 + 𝐛𝑧 )
which are relatively easy-trained, to assist the training of primary tasks
𝐢𝑡 = 𝜎(𝐖𝑖 𝐱𝑡 + 𝐐𝑖 𝐡𝑡−1 + 𝐛𝑖 )
(list-wise prediction).
𝐟 𝑡 = 𝜎(𝐖𝑓 𝐱𝑡 + 𝐐𝑓 𝐡𝑡−1 + 𝐛𝑓 ) (6)
3.3. Task relation attention networks 𝐜𝑡 = 𝐟 𝑡 ⊙ 𝐜𝑡−1 + 𝐢𝑡 ⊙ 𝐳𝑡
𝐨𝑡 = 𝜎(𝐖𝑜 𝐱𝑡 + 𝐖ℎ 𝐡𝑡−1 + 𝐛𝑜 )
In MTL, positive knowledge transfer leads to improved performance,
because: (1) more related information is incorporated into the target 𝐡𝑡 = 𝐨𝑡 ⊙ 𝑡𝑎𝑛ℎ(𝐜𝑡 ),
tasks, which benefits the feature learning process to obtain better where

4
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

• 𝐖𝑧 , 𝐖𝑖 , 𝐖𝑓 , 𝐖𝑜 ∈ R𝑈 ×𝐷 , and 𝐐𝑧 , 𝐐𝑖 , 𝐐𝑓 ∈ R𝑈 ×𝑈 are the transfor- As for the formulary definitions, assuming that the inputs of a stock
[ ]𝑇
mation matrices; list are 𝐗 = 𝐱1 , 𝐱2 , … , 𝐱𝐾 ∈ R𝐾×𝑇 , where 𝐾 is the number of stocks
• 𝐛𝑧 , 𝐛𝑖 , 𝐛𝑓 , 𝐛𝑜 ∈ R𝑈 are bias vectors; and 𝑇 is the length of sequential data, we feed the input features into
• 𝐳𝑡 is the transformed representations of inputs; the encoding module, and then we can obtain,
• 𝐢𝑡 is the input gate to control the information from inputs to [ ]
𝐇𝐿𝑆𝑇 𝑀 = LayerNorm LSTM(𝐗) + 𝐖𝑙 𝐗 ,
memories; [ ]
𝐇𝐹 𝐶𝑁 = LayerNorm FCN(𝐗) + 𝐖𝑓 𝐗 , (8)
• 𝐟 𝑡 is the forget gate to decide how much information is supposed
to be kept in memory cells; 𝐇 = 𝐇𝐿𝑆𝑇 𝑀 ∥ 𝐇𝐹 𝐶𝑁 ,
• 𝐜𝑡 is the cell state which reserves the memory information from where 𝐇 is the outputs of the shared encoding layers, ∥ represents the
inputs and last memory cell; concatenation operation, 𝐖𝑙 and 𝐖𝑓 are the linear transformations in
• 𝐨𝑡 is the output gate to regulate the amount of information that residual connections.
should be outputted in the hidden state 𝐡𝑡 . For LSTM-RNN, we use the last hidden state as the outputs of
sequential embedding model,
Benefiting from the effects of memory cells, [𝐜1 , … , 𝐜𝑡 , … , 𝐜𝑇 ]𝑇 , LSTM is
[ ]𝑇
capable of capturing the long-term dependency in the sequential data. LSTM(𝐗) = 𝐡𝑇1 , 𝐡𝑇2 , … , 𝐡𝑇𝐾 , (9)
[ 𝑇 𝑇 𝑇
]𝑇 𝐾×𝑈
4. Method where 𝐡1 , 𝐡2 , … , 𝐡𝐾 ∈ R , 𝑈 is the embedding size, (i.e., the
number of hidden units in LSTM), and 𝐡𝑇 represents the last hidden
4.1. The overall framework state.
For FCN, we use 6-layer fully-connected neural networks with acti-
The proposed method is called Multi-Task Stock Ranking (MTSR), vation of 𝑟𝑒𝑙𝑢 as the encoder,
as illustrated in Fig. 4, containing three primary modules: [ ]𝑇
FCN(𝐗) = 𝐞1 , 𝐞2 , … , 𝐞𝐾 , (10)
• Shared Encoding Layers. This module is the bottom encoding [ ]𝑇 𝐾×𝑂
where 𝐞1 , 𝐞2 , … , 𝐞𝐾 ∈ R , 𝑂 is the output size of FCN. The detailed
module shared by all tasks, aiming to discover the shared patterns
illustration of encoding module is presented in Fig. 4.
from the input features.
• Task-specific Encoding Layers with TRAN. This module is the
4.4. Task-specific encoding layers with TRAN
task-specific encoding module, where each task owns their private
encoders. The private encoders aim to learn the task-specific
Besides the bottom shared encoding layers, each task is supposed
representations. And this module is equipped with TRAN, which
to own their task-specific encoding layer, which is in order to adapt
is aimed to achieve better knowledge transfer in MTL using task
the encoded presentations to different task objectives. In MTSR, the
relations.
task-specific encoding layers are implemented with the independent
• Task-specific Prediction Layers. This module is the prediction
fully-connected neural networks with activation of 𝑡𝑎𝑛ℎ.
module, where each task owns their private prediction layers. ( )
The prediction layers are specifically designed for different task 𝐇𝑟 = 𝑡𝑎𝑛ℎ 𝐖𝑒𝑟 𝐇 + 𝐛𝑒𝑟 ,
( 𝑒 )
objectives. 𝐇𝑡 = 𝑡𝑎𝑛ℎ 𝐖𝑡 𝐇 + 𝐛𝑒𝑡 , (11)
( )
𝐇𝑒𝑟 = 𝑡𝑎𝑛ℎ 𝐖𝑒𝑒𝑥 𝐇 + 𝐛𝑒𝑒𝑥 ,
4.2. Inputs: A list of stocks where 𝐇𝑟 , 𝐇𝑡 and 𝐇𝑒𝑟 are respectively the task-specific encoded features
for ranking, trend prediction and excess return prediction tasks, 𝐖𝑟 , 𝐖𝑡
For the input features, we use the sequential features of stocks, and 𝐖𝑒𝑟 are the transformations in task-specific encoding layers.
including the trading features (opening prices, closing prices, trading And then, we use TRAN to capture task relations and achieve
volumes, etc.) and some technical indicators (MA5, MA10, MA20). In better knowledge transfer in the feature learning process. The detailed
a batch of samples, we use the list of stocks (no suspended stocks and learning process of TRAN is presented in the section of 2.3. And we
ST stocks) on one trading day. Therefore, the form of the inputs is: obtain the task-specific encoded features enhanced with knowledge
[ 𝑡 𝑇
] transfer,
𝐗𝑡 = 𝐱1𝑡 , 𝐱2𝑡 , … , 𝐱𝐾 ,
[ 𝑡−𝑇 𝑡−𝑇 +1 ]𝑇 (7) ( )
𝑡
𝐱𝑘 = 𝑥𝑘 , 𝑥𝑘 , … , 𝑥𝑡−1 , 𝑘 = 1, 2, … , 𝐾, [𝐇′𝑟 , 𝐇′𝑡 , 𝐇′𝑒𝑥 ]𝑇 = TRAN [𝐇𝑟 , 𝐇𝑡 , 𝐇𝑒𝑥 ]𝑇 , (12)
𝑘

where 𝐗𝑡 ∈ R𝐾×𝑇 ×𝐷 is the inputs of the stock list on the trading day 𝑡, where 𝐇′𝑟 , 𝐇′𝑡 and 𝐇′𝑒𝑥 are the outputs of TRAN.
and 𝐱𝑘𝑡 ∈ R𝑇 ×𝐷 is the sequential features of a single stock 𝑘.
4.5. Task-specific prediction layers
4.3. Shared encoding layers
In overall, we use the bottom-shared-and-top-private MTL frame-
The shared encoding layers are shared by all tasks, which is aimed work, where all tasks own their task-specific prediction layers based on
to discover the common patterns from the inputs. Considering there the encoded features. In the module of prediction, we construct three
are strong temporal patterns in stock data, we construct the sequential stock prediction tasks.
embedding model to capture the temporal dependencies in the stock It should be emphasized that there are the date requirements for
historical data. Since LSTM-RNN has achieved significant performances stock prediction, as follows:
in processing sequential data in several applications and demonstrated ( )
Inputs = 𝐱𝑡−𝑇 , 𝐱𝑡−𝑇 +1 , … , 𝐱𝑡−1 ,
to be effective in recent stock prediction works, we apply LSTM- (13)
Predictions = 𝐲̂ 𝑡+1 .
RNN for the sequential embedding model. Additionally, besides the
temporal patterns, the interactions between features are as well cru- As the above equations presented, if we select a certain trading date 𝑡,
cial, therefore, we construct another feature embedding model with then the inputs are based on the historical records before 𝑡, which are
Fully-connected Neural Networks (FCN), which encodes the features from 𝑡 − 𝑇 + 1 to 𝑡 − 1, and the predictions are for the date of 𝑡 + 1.
in parallel with LSTM-RNN. The final encoded representations are the Because if we make predictions for 𝑡 + 1 and implement the trading
concatenation of LSTM-RNN and FCN outputs. decisions, we are supposed to buy the selected stocks before the end of

5
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Fig. 4. The overall framework of the proposed method.

market trading on date 𝑡 and hold them to obtain the expected revenues cumulative excess returns for a future period of time, which as follows:
of 𝑡 + 1. If we buy these stocks before the market closing of 𝑡, all the
historical records we can have are the data before 𝑡. Therefore, we are [ ]𝑇
𝐘̂ 𝑟 = 𝐲̂ 𝑟1 , 𝐲̂ 𝑟2 , … , 𝐲̂ 𝑟𝐾 ,
supposed to use the historical data from 𝑡 − 𝑇 to 𝑡 − 1 to predict the (𝑁 )
values of 𝑡 + 1. ∑ (15)
𝐲̂ 𝑟𝑘 = Rank 𝐸𝑅𝑡+𝑖 𝑘
, 𝑘 = 1, 2, … , 𝐾,
𝑖=1
4.5.1. Primary task: List-wise stock ranking
where 𝐲̂ 𝑟𝑘 ∈ R represents the predicted ranking values for stock 𝑘, and
This task is regarded as the primary task in MTSR, which aims to
𝐸𝑅𝑡+𝑖 is the expected excess returns of stock 𝑘 on date 𝑡 + 𝑖. The excess
predict the future ranking results (in terms of the cumulative excess 𝑘
returns represents the individual stock profits which exceed the market
returns for a future period of time) for a list of stocks at the same time.
average returns, as follows:

1 ∑ 𝑡
𝐾
𝐘̂ 𝑎𝑟 = 𝐖𝑎𝑟 𝐇′𝑟 + 𝐛𝑎𝑟 ,
( ) 𝐸𝑅𝑡𝑘 = 𝑅𝑡𝑘 − 𝑅̄ 𝑡𝑀 = 𝑅𝑡𝑘 − 𝑅 , 𝑘 = 1, 2, … , 𝐾. (16)
𝐘̂ 𝑠 = 𝜎 𝐖𝑠 𝐇′ + 𝐛𝑠 , (14) 𝐾 𝑘=1 𝑘
𝑟 𝑟 𝑟 𝑟
𝐘̂ 𝑟 = 𝐘̂ 𝑠𝑟 + 𝑒𝑝𝑠 × 𝐘̂ 𝑠𝑟 ,
𝑒𝑝𝑠 ∈ 𝑟𝑎𝑛𝑑(−1, 1), where 𝑅𝑡𝑘 is the net returns of stock 𝑘 on date 𝑡, and 𝑅𝑡𝑀 is the average
[ ]𝑇 return of stock markets on date 𝑡.
where 𝐘̂ 𝑟 = 𝐲̂ 𝑟1 , 𝐲̂ 𝑟2 , … , 𝐲̂ 𝑟𝐾 ∈ R𝐾×1 is the list-wise ranking prediction As for the loss function of list-wise stock ranking, we use the
for 𝐾 stocks. The prediction layers are implemented with two inde- additive inverse of Information Ratio (IC) as the objective to optimize.
pendent single-layer neural networks, respectively predicting the mean IC is the Pearson correlation between the predicted ranking values
value and standard deviation of ranking results, then combining them and the real ranking values, widely used as a metric to evaluate the
with 𝐘̂ = 𝝁̂ + 𝑒𝑝𝑠 × 𝝈, ̂ 𝑒𝑝𝑠 is a randomly sampled value between −1 and performances of stock ranking. The normal range of IC is between −1
1, similar to the process of Variational Auto-Encoder (VAE). Because and 1, and the closer to 1, the better. Since we aim to minimize the
of the strong complexity of stock ranking, short-term small fluctuations loss, therefore, we use the additive inverse of IC as the loss function. In
of stocks are likely to cause over-fitting. Such forecasting methods with the proposed method, the aim of stock ranking is based on the levels of
uncertainties can better tolerate such fluctuations, thereby reducing the excess returns, which means that the more excess returns a stock has,
risk of over-fitting. the more higher ranking value it owns.
And for the detailed predicted objectives, the ranking task aims ( )
to predict the ranking results for a list of stocks in terms of their 𝑙𝑟 = −IC = −corr 𝐘̂ 𝑟 , 𝐘𝑟 , (17)

6
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

where 𝑙𝑟𝑎𝑛𝑘 ∈ R is the additive inverse of the Pearson correlation 4.7. Enhance list-wise stock ranking task with auxiliary tasks
between predicted ranking vector and real ranking vector.
As we discussed in the previous section, there are both advan-
4.5.2. Auxiliary task I: Point-wise stock trend prediction tages and disadvantages in three approaches of stock prediction. To
This task is an auxiliary task, which aims to predict the future trends enhance the learning process of list-wise stock ranking, we construct
(binary classification: rise or fall) for every individual stock. two point-wise prediction tasks, as illustrated in Fig. 4.
( ) First, we discuss about the reason why we did not use the task of
𝐘̂ 𝑡 = Sof tmax 𝐖𝑡 𝐇′𝑡 + 𝐛𝑡 , (18) pair-wise stock prediction. There are two primary concerns about it:
[ ]𝑇 (1) As an auxiliary task, there are more training complexities in pair-
where 𝐘̂ 𝑡 = 𝐲̂ 𝑡1 , 𝐲̂ 𝑡2 , … , 𝐲̂ 𝑡𝐾 ∈ R𝐾×2 is the trend predictions. The pre- wise prediction than point-wise prediction. Our aim of choosing the
diction layer for point-wise stock trend prediction is implemented with auxiliary tasks with less training complexities is to guide the learning
a single-layer fully-connected neural network with 𝑆𝑜𝑓 𝑡𝑚𝑎𝑥 activation. of primary tasks, therefore, pair-wise prediction is not suitable for our
For the detailed predicted objectives for stock trend prediction, it aim. (2) Pair-wise prediction needs to split samples into several pairs.
aims to predict the future trends of the cumulative stock returns for a In the case that our primary task is list-wise, this way brings more
future period of time, as follows: difficulties to the sample splitting process, and more computational
[ ]𝑇 complexities. Therefore, considering these two concerns, we prefer to
𝐘̂ 𝑡 = 𝐲̂ 𝑡1 , 𝐲̂ 𝑡2 , … , 𝐲̂ 𝑡𝐾 ,
(𝑁 ) choose point-wise prediction as auxiliary tasks.
∑ (19) Then, we discuss about what role the auxiliary tasks play in this
𝐲̂ 𝑡𝑘 = Trend 𝑅𝑡+𝑖 𝑘
, 𝑘 = 1, 2, … , 𝐾. method. There are two primary aspects of their contributions to the
𝑖=1
primary task: (1) Act as learning guider. As we discussed before, our
For the binary classification task of stock trend prediction, Binary primary task, list-wise stock ranking, is advanced in stock selection
Cross-Entropy (BCE) is used as loss function. however it brings a lot of learning complexities to the model because
there will be a small amount of samples in the list-wise splitting. For
1 ∑
𝐾
( )
𝑙𝑡 = BCE 𝐲̂ 𝑡𝑘 , 𝐲𝑡𝑘 , (20) point-wise prediction, it is much easier to be trained, because, facing
𝐾 𝑘=1 with the same data, it will contain a much larger number of samples
where 𝐲̂ 𝑟𝑘 and 𝐲𝑟𝑘 are respectively the predicted trend probability and in the point-wise splitting. With the assistance of MTL, the addition of
the real trend probability of stock 𝑘. easily trained auxiliary tasks acts as learning guider to help the learning
of primary task to achieve satisfying performances. (2) Bring auxiliary
knowledge for stock ranking. There is strong relevance between primary
4.5.3. Auxiliary task II: Point-wise excess return regression
task and auxiliary tasks, e.g., the auxiliary task of trend prediction is
This is another auxiliary task, which aims to predict the future
aimed to estimate whether stocks will rise or fall in the future, while
excess returns (regression) for every individual stock.
for stock ranking, the rising stocks are supposed to have higher ranks
𝐘̂ 𝑒 = 𝐖𝑒 𝐇′𝑒 + 𝐛𝑒 , (21) than the failing stocks. The predicted aims of the constructed auxiliary
[ ]𝑇 tasks are consistent with the one of primary task, as a result, in the
where 𝐘̂ 𝑒 = 𝐲̂ 𝑒1 , 𝐲̂ 𝑒2 , … , 𝐲̂ 𝑒𝐾 ∈ R 𝐾×1 is the excess return predictions. framework of MTL, they can transfer additional knowledge to primary
The prediction layer for point-wise excess return regression is imple- task to help achieve better ranking performances.
mented with a single-layer fully-connected neural network with linear
activation. 5. Experiments
For the detailed predicted objectives for stock excess return pre-
To the best of our knowledge, the proposed method is the first
diction, it aims to predict the future values of the cumulative excess
deep learning based work to incorporate Multi-Task Learning into the
returns for a future period of time, as follows:
problem of stock ranking prediction. To evaluate the performances of
[ ]𝑇
𝐘̂ 𝑒 = 𝐲̂ 𝑒1 , 𝐲̂ 𝑒2 , … , 𝐲̂ 𝑒𝐾 , the proposed method, back-testing experiments are conducted on the
(𝑁 ) real-world stock datasets, including the data of Chinese stock market
∑ (22)
𝐲̂ 𝑒𝑘 = Reg 𝐸𝑅𝑘𝑡+𝑖 , 𝑘 = 1, 2, … , 𝐾, and the data of American stock market. We choose multiple baseline
𝑖=1 methods, including some state-of-the-art stock prediction methods, to
For the regression task of excess return prediction, we use Mean compare with MTSR to demonstrate its superiority.
Squared Error (MSE) as its loss function.
5.1. Dataset
1 ∑
𝐾
( )
𝑙𝑒 = MSE 𝐲̂ 𝑒𝑘 , 𝐲𝑒𝑘 , (23) Most of existing works about stock predictions evaluate on dozens
𝐾 𝑘=1
of stocks and rarely make predictions for a large amount of stocks.
where 𝐲̂ 𝑒𝑘 and 𝐲𝑒𝑘 are respectively the predicted excess return and the Therefore, we aim to construct experiments on the real-world datasets
real excess return of stock 𝑘. containing more than 3000 stocks. Specifically, we collect the daily
We did not use the approach of pair-wise stock prediction in the transaction data respectively from the Chinese stock market and the
MTL framework, and we will discuss about it in the later subsection. American stock market. Here are the detailed introductions about the
The model structure of multi-task stock ranking is also presented in datasets.
Fig. 4.
5.1.1. Dataset of Chinese stock market
From the Chinese stock market, we collect the data of 3815 stocks
4.6. Joint loss function
in total from 2016–01 to 2019–12. The transaction records include
the prices, trading volumes and amounts, etc. Besides the features
In the framework of MTL, the learning process of multiple tasks is
about transactions, we add three extra features including MA5 (Moving
guided by the joint loss function. The joint loss function is the weighted Average over 5 trading days), MA10 and MA20, which respectively rep-
sum of three task-specific losses, defined as follows: resent the weekly, half-monthly and monthly trends. In pre-processing,
𝐿 = 𝛽𝑟 × 𝑙𝑟 + 𝛽𝑡 × 𝑙𝑡 + 𝛽𝑒𝑥 × 𝑙𝑒 . (24) we drop the ST (Special Treatment) stocks, suspended stocks and
the stocks which have been traded on less than 95% of the trading
All tasks are learned simultaneously with the optimization of joint loss days from 2016–01 to 2019–12. Finally, we obtain 3315 stocks for
function, and the weights of each task losses are hyper-parameters. experiments (see Table 1).

7
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Table 1 • MTSR wo TRAN: This is the method equipped with MTL but
Introduction of the stock datasets.
not considering the enhanced knowledge transfer, which is our
Dataset Num of stocks Date period Trading days
proposed method MTSR without TRAN.
Chinese 3315 2016-01 to 2019–12 975 • MTSR: This is our proposed method MTSR.
American 3678 2013-01 to 2017–12 1259

5.3. Experimental settings

5.1.2. Dataset of American stock market Here are the implementation details of experiments.
From the American stock market (NASDAQ and NYSE), we collect
the data of 7915 stocks in total from 2013–01 to 2017–12. The form of
5.3.1. Parameter settings
transactions records is similar to the one of Chinese market, including
the basic prices, trading volumes and amounts, etc. We also add MA5, All of the mentioned methods, including MTSR and other base-
MA10 and MA20 into the feature dimensions. In pre-processing, we line methods, are implemented with Tensorflow (Abadi et al., 2016).
drop the stocks which have been traded on less than 95% of the trading In some related works, the hyper-parameter selecting processes are
days from 2013–01 to 2017–12, and finally, we obtain 3697 stocks for usually equipped with optimization algorithms (Hu & Tan, 2015; Liu,
experiments (see Table 1). Zheng, & Tan, 2015; Zheng & Tan, 2013; Zhou & Tan, 2011), which
usually bring in extra costs for computition efficiency. In this paper,
5.2. Baseline methods we perform grid searching to select the optimal hyper-parameters for
all methods.
To evaluate the performances of the proposed method, we compare
it with the following baseline methods: • For LSTM, we tune two hyper-parameters for it, respectively the
length of sequential inputs 𝑇 within {20, 40, 60} and the number
• LSTM (Jin, Yang, & Liu, 2020): This method is based on the of hidden units within {16, 32, 64, 128}.
vanilla LSTM-RNN, which processes on the stock data to dis- • For FCN, we tune two hyper-parameters for it, respectively the
cover their sequential patterns to predict their future stock excess number of layers within {3, 4, 5, 6, 7} and the number of hidden
returns, serving as a baseline method with point-wise stock pre-
units within {64, 128, 256, 512, 1024}.
diction. The input features are firstly fed into a LSTM-RNN layer
• For SFM, we follow the original settings in the works of Zhang
for encoding, and the encoded features are fed into the final FC
et al. (2017), tuning the number of frequencies within {5, 10, 15}
layer for prediction.
and the number of hidden units within {10, 20, 30}.
• FCN: This method is based on the classical Fully-Connected Neu-
• For ALSTM, we tune two hyper-parameters for it, respectively the
ral Networks to predict the future stock excess returns, serving as
a baseline method with point-wise stock prediction. The collected length of sequential inputs 𝑇 within {20, 40, 60} and the number
sequential features for inputs are flattened and fed into an 5-layer of hidden units 𝑈 within {16, 32, 64, 128}.
FCN for encoding. The encoded features are fed into the final FC • For MSJF, we follow the original settings in the works of Ma and
layer with linear transformation to make the predictions. Tan (2020c), tuning the number of hidden units within {16, 32,
• SFM (Zhang et al., 2017): This method is the state-of-the-art 64, 128}.
point-wise stock prediction method. It takes the historical closing • For DSR, we follow the original settings in the works of Zhang
prices as inputs and decomposes prices into the signals of different and Tan (2018), tuning the number of hidden units within {32,
frequencies with a Discrete Fourier Transform (DFT). And then, it 64, 128, 256}.
feeds the DFT coefficients into an extended LSTM with separate • For STSR, we tune the numbers of hidden units in its encoding
memory states for different frequencies to learn the frequency- layers.
aware sequential embeddings, and the embeddings are fed into • For MTSR without TRAN and MTSR, besides the tuning for
the final FC layer for prediction. This method serves as a baseline encoding layers, we tune the coefficients of task losses within
method with point-wise stock prediction. {0.1, 0.3, 0.5, 0.7, 1.0}.
• ALSTM (Qin et al., 2017): This method serves as another state-of-
the-art point-wise stock prediction method. It enhanced the basic The training processes of all methods are guided by the optimizer
LSTM model with temporal attention aggregation layers. of Adam (Kingma & Ba, 2014), and the initial learning rate is set as
• MSJF (Ma & Tan, 2020c): This method is the state-of-the-art MTL 0.001.
based point-wise stock prediction method. It processes the se-
quential features of multiple stocks and predict their future prices 5.3.2. Rolling training, validation and testing
simultaneously with MTL. This method considers the correlations To better discover the sequential patterns in stock data, the learning
between different stocks using the MTL framework, however, it process is guided with the approach of rolling training, validation
still focuses on the point-wise method and does not consider the and testing. The procedures are illustrated in Fig. 5, described as: (1)
enhancements of auxiliary stock prediction tasks. This method
Training: select the samples in a period of time (in terms of trading
serves as a baseline method with point-wise prediction and MTL.
days) for training within a number of epochs; (2) Validation: select
• DSR (Zhang & Tan, 2018): This method is the state-of-the-art
the samples in the period following the training window for validation.
list-wise stock ranking prediction method. It takes the historical
If there are no significant declines of validation losses in a number of
trading records (trading prices, trading amount and volumes) as
epochs, then stop the training process; (3) Testing: select the samples
inputs and uses LSTM-RNN to encode the sequential features. And
finally, it feeds the encoded features into a FC layer to make list- in the period following the validation period for testing; (4) Rolling
wise stock ranking predictions. This method serves as a baseline the window forwards: when the procedure of testing ends up, put
method with list-wise stock ranking prediction. the validation samples into the training dataset and roll the window
• STSR: This is the method removing MTL and TRAN from the forwards; (5) Repeat: repeat the training process in procedure (1).
proposed method, performing as a Single-Task Stock Ranking The lengths of training, validation and testing periods are hyper-
method. This method only performs the list-wise ranking predic- parameters as well, we also use grid searching to tune their values,
tion task, which ignores the enhancements of auxiliary tasks. It tuning the training window length within {120, 240, 360, 480}, the
serves as a baseline to demonstrate the effectiveness of MTL in validation window length within {10, 20, 40, 60} and the testing
our method. window length within {10, 20, 40, 60}.

8
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Fig. 5. Illustration of rolling training, validation and testing.

5.3.3. The window length of predicted objectives • Wining Rate (WR): the winning rate of portfolios gaining positive
As presented in the section of 3.5, the constructed tasks aim to absolute returns, which is a widely-used metric to evaluate the
predict the cumulative values for a future period of time with the length profitability of portfolios.
of 𝐹 . And this window length is equal to the trading frequency in the 𝑁𝑟>0
WR = × 100%, (27)
back-testing strategy. For example, if we aim to adjust the positions 𝑁𝑑
every 𝐹 trading days in the trading strategy, then the window length where 𝑁𝑟>0 is the number of days when the portfolio can gain
of predicted objectives is 𝐹 as well. In the implementation, the length positive returns, and 𝑁𝑑 is the total number of trading days in
is assigned as 10 (trading days). the portfolio.
• Profit & Loss Ratio (PLR): the ratio of all profits and all losses,
5.4. Back-testing settings also denoting the profitability of portfolios.
∑ 𝑁𝑑
𝑟
𝑛=1,𝑟 >0 𝑛
To comprehensively evaluate the performances of MTSR, we con- PLR = ∑𝑁 𝑛 × 100%, (28)
𝑑
|𝑟 |
𝑛=1,𝑟 <0 𝑛
struct the back-testing experiments for all the stock prediction methods 𝑛

with a trading strategy. Here are the detailed introductions about the where 𝑟𝑛 is the portfolio return of 𝑛th day.
strategy: • Actual Wining Rate (AWR): the product of WR and PLR.

• Trading period: 2018–01 to 2019–12 (Chinese), 2016–01 to 2017– AWR = WR × PLR. (29)
12 (American); • Cumulative Returns (CR): the cumulative returns of portfolios in
• Range of stock selection: All the predicted stocks; the whole investment process.
• Selected stocks: Top 10 stocks; 𝑁𝑑
• Position distribution: equal ratios of the positions for the selected ∑
CR = 𝑟𝑛 × 100%. (30)
stocks; 𝑛=1
• Trading frequency and costs: adjust the stock positions every 10
• Annual Return (AR): the annualized returns of trading strategies,
trading days, and the trading costs are assigned as 0.1%;
denoting the profitability of portfolios. In our experiments, we
• Trading strategy: buy the selected stocks with equal positions and
assign the number of trading days in one year as 240.
hold for 10 trading days, after that, sell them and buy another 10
∑𝑁𝑑
selected stocks in the next term, and repeat this strategy until the 𝑟𝑛
AR = 𝑛=1 × 240 × 100%. (31)
end. 𝑁𝑑

The back-testing strategy is a buy-hold-sell strategy. It means when you • Max Drawdown (MDD): the maximum short-term losses suffered
buy the selected stocks at the beginning date, you are supposed to hold in the whole investment process.
them for a number of days. When it is time to adjust the positions, you
MDD = max CR𝑖 − CR𝑗 , (32)
should sell the holding stocks and buy the selected stocks for the next 𝑖<𝑗

trading period. where CR𝑖 the cumulative returns till time 𝑖.

• Volatility (VOL): the annualized volatility of portfolio returns,
5.5. Evaluation metrics indicating the stability of portfolios.
√
VOL = 𝜎(𝑅) × 240. (33)
We use multiple metrics to evaluate the performances of all meth-
• Sharpe Ratio (Sharpe): a widely-used metric to measure the in-
ods, including: vestment quality of portfolios, which is the ratio of the average
and standard deviation of portfolio returns, considering both of
• Information Coefficient (IC): the Pearson correlation between profitability and investment risk.
predicted ranking scores and real ranking scores, widely used to
𝜇(𝑅) √
evaluate the performance of stock ranking. Sharpe = × 240, (34)
( ) 𝜎(𝑅)
IC = corr 𝐘̂ 𝑟 , 𝐘𝑟 . (25) where 𝑅 = (𝑟1 , 𝑟2 , … , 𝑟𝑁𝑑 ) is the portfolio returns, and the annu-
alized values of SR are used in the experiments.
• Rank Information Coefficient (RankIC): the Pearson correlation • Calmar Ratio (Calmar): another metric to measure the quality of
between predicted ranks and real ranks. portfolios, which is the ratio of average returns and MDD.
( ) 𝜇(𝑅)
RankIC = corr Rank(𝐘̂ 𝑟 ), Rank(𝐘𝑟 ) . (26) Calmar = . (35)
𝑀𝐷𝐷

9
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Table 2
Performance comparison in Chinese stock markets. All the methods have been run 10 times with different random seeds. The values of metrics are presented in the form of
mean±std.
LSTM FCN SFM ALSTM MSJF DSR STSR MTSR wo TRAN MTSR
IC −0.69 ± 0.60% −0.53 ± 0.68% 2.12 ± 0.77% 1.57 ± 0.71% 1.93 ± 0.62% 2.70 ± 0.80% 3.53 ± 0.52% 3.75 ± 0.54% 3.60 ± 0.53%
RankIC 0.96 ± 0.45% 1.27 ± 0.43% 2.33 ± 0.46% 1.63 ± 0.40% 2.11 ± 0.41% 3.01 ± 0.52% 3.43 ± 0.54% 3.98 ± 0.47% 3.62 ± 0.51%
WR 50.99 ± 1.02% 50.11 ± 1.03% 53.64 ± 0.79% 53.20 ± 1.01% 53.20 ± 0.93% 55.85 ± 0.74% 52.76 ± 0.97% 53.42 ± 0.72% 56.73 ± 0.71%
PLR 0.983 ± 0.032 1.025 ± 0.025 1.160 ± 0.041 1.072 ± 0.035 1.079 ± 0.032 1.149 ± 0.035 1.146 ± 0.041 1.167 ± 0.031 1.256 ± 0.041
AWR 50.12 ± 2.35% 51.37 ± 1.97% 62.23 ± 2.11% 57.01 ± 2.71% 57.42 ± 2.69% 64.20 ± 3.23% 60.45 ± 2.49% 62.36 ± 3.16% 71.28 ± 2.38%
CR −4.88 ± 7.45% 6.86 ± 6.29% 44.19 ± 7.54% 19.50 ± 8.29% 21.64 ± 6.77% 38.80 ± 6.20% 36.28 ± 10.26% 41.36 ± 9.74% 61.48 ± 7.68%
AR −2.59 ± 4.31% 3.64 ± 4.72% 23.41 ± 5.30% 10.33 ± 5.09% 11.46 ± 4.58% 20.56 ± 3.94% 19.22 ± 3.57% 21.91 ± 3.99% 32.57 ± 5.26%
MDD −27.11 ± 3.36% −36.11 ± 2.26% −41.60 ± 2.76% −42.12 ± 2.79% −41.80 ± 3.23% −35.94 ± 2.81% −29.34 ± 2.66% −23.15 ± 3.11% −25.78 ± 2.61%
VOL 25.64 ± 0.31% 25.82 ± 0.40% 26.89 ± 0.35% 25.21 ± 0.34% 25.23 ± 0.34% 24.72 ± 0.38% 24.20 ± 0.30% 24.56 ± 0.40% 24.06 ± 0.28%
Sharpe −0.101 ± 0.165 0.141 ± 0.208 0.871 ± 0.195 0.410 ± 0.139 0.454 ± 0.179 0.831 ± 0.206 0.794 ± 0.153 0.892 ± 0.155 1.354 ± 0.184
Calmar −0.095 ± 0.183 0.101 ± 0.199 0.563 ± 0.132 0.245 ± 0.171 0.274 ± 0.141 0.572 ± 0.200 0.655 ± 0.209 0.946 ± 0.157 1.263 ± 0.203
EWR 51.88 ± 0.51% 50.11 ± 0.60% 53.64 ± 0.71% 51.88 ± 0.43% 52.54 ± 0.55% 54.08 ± 0.48% 52.32 ± 0.45% 54.75 ± 0.61% 53.64 ± 0.69%
EPLR 1.025 ± 0.052 1.099 ± 0.051 1.326 ± 0.054 1.174 ± 0.064 1.185 ± 0.053 1.317 ± 0.066 1.298 ± 0.057 1.334 ± 0.054 1.499 ± 0.067
EAWR 53.18 ± 3.27% 55.07 ± 3.02% 71.11 ± 3.14% 60.91 ± 2.79% 62.25 ± 3.72% 71.20 ± 4.28% 67.91 ± 3.74% 73.04 ± 3.70% 80.43 ± 2.78%
EAR 2.47 ± 4.33% 8.69 ± 4.41% 28.47 ± 4.35% 15.39 ± 5.26% 16.52 ± 4.84% 25.61 ± 5.08% 24.28 ± 3.82% 26.97 ± 3.92% 37.63 ± 4.01%
IR 0.147 ± 0.303 0.540 ± 0.239 1.631 ± 0.244 0.934 ± 0.317 0.995 ± 0.261 1.624 ± 0.311 1.524 ± 0.213 1.733 ± 0.244 2.345 ± 0.234

Table 3
Performance comparison in American stock markets. All the methods have been run 10 times with different random seeds. The values of metrics are presented in the form of
mean±std.
LSTM FCN SFM ALSTM MSJF DSR STSR MTSR wo TRAN MTSR
IC 0.70 ± 0.35% 0.65 ± 0.33% 1.23 ± 0.39% 1.06 ± 0.30% 1.14 ± 0.33% 1.75 ± 0.29% 2.37 ± 0.39% 2.70 ± 0.32% 2.55 ± 0.37%
RankIC 0.04 ± 0.24% 0.69 ± 0.26% 0.68 ± 0.28% 0.47 ± 0.31% 0.56 ± 0.19% 0.40 ± 0.23% 1.52 ± 0.20% 1.78 ± 0.26% 1.70 ± 0.20%
WR 51.61 ± 1.38% 52.90 ± 0.99% 52.47 ± 1.23% 52.04 ± 0.92% 52.04 ± 1.26% 54.19 ± 1.30% 53.98 ± 1.31% 58.92 ± 0.92% 58.71 ± 1.29%
PLR 1.032 ± 0.190 1.065 ± 0.155 1.161 ± 0.198 1.062 ± 0.172 1.064 ± 0.151 1.562 ± 0.198 1.784 ± 0.154 1.953 ± 0.125 1.916 ± 0.131
AWR 53.27 ± 8.58% 56.34 ± 10.49% 60.91 ± 10.85% 55.29 ± 9.38% 55.35 ± 12.57% 84.65 ± 11.31% 96.30 ± 9.10% 115.10 ± 10.63% 112.51 ± 10.29%
CR 5.31 ± 35.06% 10.83 ± 39.48% 26.55 ± 32.93% 10.19 ± 30.95% 10.35 ± 37.95% 124.75 ± 32.85% 164.70 ± 29.40% 178.98 ± 27.67% 183.34 ± 26.40%
AR 2.74 ± 12.91% 5.59 ± 17.17% 13.70 ± 18.03% 5.26 ± 14.48% 5.34 ± 15.09% 64.39 ± 15.14% 85.00 ± 16.94% 92.38 ± 18.60% 94.63 ± 17.34%
MDD −14.54 ± 1.05% −17.42 ± 1.07% −11.58 ± 0.87% −11.74 ± 0.92% −11.58 ± 0.81% −15.70 ± 0.68% −10.80 ± 0.90% −12.70 ± 0.67% −13.63 ± 0.89%
VOL 14.80 ± 2.13% 15.33 ± 2.36% 18.19 ± 2.45% 14.84 ± 1.74% 14.80 ± 1.70% 24.99 ± 2.10% 25.10 ± 2.13% 24.28 ± 2.56% 25.98 ± 1.80%
Sharpe 0.185 ± 0.781 0.365 ± 0.596 0.753 ± 0.525 0.354 ± 0.704 0.361 ± 0.646 2.577 ± 0.777 3.387 ± 0.661 3.805 ± 0.689 3.642 ± 0.753
Calmar 0.189 ± 1.497 0.321 ± 1.053 1.183 ± 1.400 0.448 ± 1.417 0.461 ± 1.558 4.100 ± 1.446 7.873 ± 1.118 7.272 ± 1.402 6.943 ± 1.152
EWR 45.81 ± 2.02% 48.82 ± 2.17% 45.59 ± 1.70% 46.24 ± 1.67% 46.24 ± 1.89% 53.33 ± 1.61% 53.98 ± 1.53% 56.77 ± 1.56% 56.56 ± 1.72%
EPLR 0.838 ± 0.183 0.862 ± 0.138 0.967 ± 0.186 0.866 ± 0.202 0.867 ± 0.216 1.469 ± 0.138 1.694 ± 0.150 1.856 ± 0.199 1.817 ± 0.150
EAWR 38.39 ± 11.27% 42.06 ± 12.94% 44.10 ± 12.48% 40.02 ± 12.75% 40.07 ± 12.16% 78.33 ± 13.45% 91.46 ± 11.74% 105.35 ± 9.17% 102.75 ± 11.52%
EAR −13.71 ± 18.73% −10.87 ± 17.64% −2.75 ± 20.85% −11.20 ± 15.91% −11.11 ± 20.48% 47.93 ± 20.28% 68.55 ± 15.55% 75.92 ± 13.11% 78.17 ± 14.97%
IR −0.979 ± 0.645 −0.865 ± 0.769 −0.158 ± 0.734 −0.796 ± 0.803 −0.790 ± 0.958 2.199 ± 0.996 3.034 ± 0.668 3.492 ± 0.889 3.269 ± 0.931

• Excess Wining Rate (EWR): the winning rate of portfolios gaining 5.6. Performance comparison
positive excess returns, i.e., outperforming the market index.
𝑁𝑟>𝑟𝑀 The performance comparisons are presented in Tables 2 and 3,
EWR = × 100%, (36) respectively corresponding to the results in Chinese stock markets and
𝑁𝑑
American stock markets. The backtesting performances are illustrated
where 𝑁𝑟>𝑟𝑀 is the number of days when the portfolio can gain in Figs. 6 and 7. It should be emphasized that there are only point-wise
positive excess returns. 𝑟𝑀 indicates the market returns, usually binary trend prediction tasks in LSTM, FCN, SFM, ALSTM and MSJF.
using the index returns. In our experiments, we use SH and For DSR and STSR, there are only ranking tasks.
NASDAQ as the benchmark indexes of respectively Chinese and
American markets. 5.6.1. Overall comparison
• Excess Profit & Loss Ratio (EPLR): the ratio of all positive excess As presented in the tables, the comparisons in Chinese markets
returns and all negative excess returns. demonstrate that the proposed method MTSR can outperform the other
∑𝑁𝑑 baseline methods in most metrics. As the comparisons in American
𝑛=1,𝑟 >𝑟
𝑟𝑛 − 𝑟𝑀 markets, MTSR wo TRAN performs best in 11 metrics and MTSR
EPLR = ∑𝑁 𝑛 𝑀 × 100%, (37) performs best in 4 metrics, both outperforming the baseline methods.
𝑑
𝑛=1,𝑟 <𝑟
|𝑟𝑛 − 𝑟𝑀 |
𝑛 𝑀 In Chinese stock markets, MTSR can outperform all the state-of-the-
where 𝑟𝑛 is the portfolio return of 𝑛th day. art methods. It outperforms the basic LSTM and FCN with significant
• Excess Actual Wining Rate (EAWR): the product of EWR and improvements in several metrics, especially in IC, RankIC, Sharpe,
EPLR. Calmar and IR. Compared with the other point-wise methods, including
SFM, ALSTM and MSJF, the proposed MTSR outperforms them with
EAWR = EWR × EPLR. (38) the relative improvements of 70.2%, 128.9% and 86.4% regarding to
IC, 55.8%, 122.6% and 71.0% regarding to RankIC. Compared with
• Excess Annual Return (EAR): the annualized excess returns, which
the list-wise method DSR, it gains the relative improvements of 32.1%
are the returns exceeding the benchmark (market) returns, also
in IC and 18.2% in RankIC. In American stock markets, MTSR can
denoting the profitability of portfolios.
also outperform all the state-of-the-art methods with significant relative
EAR = AR − AR𝑀 , (39) improvements. These demonstrate the effectiveness of the proposed
method.
where AR𝑀 indicates the market annual returns.
• Information Ratio (IR): a widely-used metric to measure whether 5.6.2. Backtesting performance comparison
portfolios can stably outperform the market, which is the ratio of The back-testing performances, as well as some critical evaluation
the average and standard deviation of portfolio excess returns. metrics, are illustrated in Figs. 6 and 7.
𝜇(𝑅 − 𝑅𝑀 ) √ As presented in Fig. 6, the proposed method MTSR obtains the best
IR = × 240, (40) cumulative returns in both Chinese and American markets, significantly
𝜎(𝑅 − 𝑅𝑀 )
outperforming the other methods. And in Fig. 7, there are obvious
where 𝑅 − 𝑅𝑀 = (𝑟1 − 𝑟𝑀 1
, 𝑟2 − 𝑟𝑀
2
, … , 𝑟𝑀
𝑁𝑑
) is the portfolio advantages of MTSR over the others in terms of profitability and
excess returns, and the annualized values of IR are used in our investment quality metrics, especially in Chinese markets. Besides, the
experiments. backtesting experiments in real-world stock datasets demonstrate the
effectiveness of MTSR in stock investments.

10
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Fig. 6. Backtesting performance illustration of all methods in Chinese and American stock markets.

Fig. 7. Evaluation performance comparison of all methods.

Additionally, compared with STSR and MTSR wo TRAN, the pro- The primary advantages of list-wise stock ranking methods are in
posed MTSR achieve better backtesting performances in overall, which the following two aspects:
demonstrate the enhancements of MTL and TRAN in our method.
• List-wise ranking methods aim to make predictions for a list of
6. Analysis of key components stocks simultaneously, considering both the interactions and com-
parisons between them. In contrast, point-wise based methods
6.1. Effects of list-wise stock ranking make predictions for individual stocks separately.
• The target of stock selection mainly focuses on determining which
In this paper, list-wise stock ranking prediction is set as the primary stocks should be invested, usually selecting the stocks with the
task, instead of point-wise or pair-wise approaches. Compared with the best expected future revenues. In another word, it only needs to
way of point-wise prediction, list-wise stock ranking prediction aim to determine the ranking results of all stocks to know which are
directly give the future ranking predictions for a list of stocks, which is the top stocks in the market that should be selected, rather than
the most consistent with the target of stock selection. However, most focusing on predicting the specific values of stocks. In fact, there
of existing stock predictions are still focusing on the way of point- are more difficulties in predicting specific values than predicting
wise prediction. As for the approaches with pair-wise prediction, their rankings, according to the Efficient Market Hypothesis. Therefore,
disadvantages have been discussed in the previous section. the direct way of predicting rankings is more efficient and easier
In our experiments, MTSR is compared with several state-of-the-art than the point-wise way of predicting the future values.
baselines with point-wise predictions. From the comparisons in Tables 2
and 3, it can be observed that the basic point-wise solutions LSTM and 6.2. Effects of MTL
FCN can hardly provide effective predictions when facing thousands of
stocks. SFM, ALSTM and MSJF can outperform the basic ones to some MTL is one of the key components in the proposed MTSR. MTL is
extent, but they fail to catch up the list-wise solutions, including DSR, able to use auxiliary tasks to enhance the primary tasks. Especially in
STSR, MTSR wo TRAN and MTSR. And in the backtesting comparisons the problem of stock prediction, the learning difficulties of list-wise
in Fig. 6, the list-wise ranking methods obtain more investment profits stock ranking can be alleviated by the easily-trained auxiliary tasks,
than the point-wise ones, especially in American markets. This indicates which act as learning guider.
the superiority of list-wise ranking solutions in dealing with a large From the performances comparisons in Tables 2 and 3, here are the
number of stocks. observations:

11
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Fig. 8. Illustration of the task weights in knowledge transfer, learned by TRAN.

• All the MTL based methods, MTSR and MTSR wo TRAN, outper- 6.4. Analysis of MTSR in two stock markets
forms the methods with single-task learning. This indicates the
effectiveness of MTL in the stock prediction domains. As presented in Tables 2 and 3, MTSR performs the best in Chi-
• In the comparison between STSR and MTSR wo TRAN, the MTL nese stock markets regarding to most evaluation metrics. While in
based method outperforms the one with only single task, which American stock markets, MTSR generally works better than most base-
demonstrates that the addition of MTL can further improve the line solutions expect MTSR wo TRAN. Regarding to the backtesting
stock prediction performance of the proposed method. performances in Fig. 6, MTSR obtains the best cumulative returns in
both Chinese and American markets. However, there is more revenue
In the backtesting performance comparisons, it can be observed volatility in Chinese markets, which is caused by the overall trading
that MTSR and MTSR wo TRAN achieve better profits than the other traits of markets.
methods, and reveal superiority in all trading metrics. As well, MTSR In the illustrated comparison in Fig. 7, it can be observed that
wo TRAN still outperforms STSR in backtesting. These observations are MTSR obtains more effective stock ranking signals (IC and RankIC) in
consistent with the previous ones, which indicate the effectiveness of Chinese markets, compared with its performances in American markets.
MTL in stock prediction and the proposed method. This may be caused by the weaker efficiency of Chinese stock markets,
The primary contributions of MTL can be summarized in two as- therefore it is easier for Alpha discovery (stock selection). Additionally,
pects: the relative improvements of MTSR over other baseline methods are
more significant in Chinese markets, especially in the profitability
• Act as learning guider. The primary task of list-wise stock ranking and investment quality evaluations. Compared with American markets,
is advanced in stock selection, however, it brings in a lot of learn- there are much more complex information pervading in Chinese mar-
ing difficulties to the model. Within the joint-learning framework kets, therefore, its trading patterns are more difficult for all methods
of MTL, the addition of easily-trained auxiliary tasks can act as to extract. However, MTSR still achieves the best performances, which
learning guider to help the learning of our primary task to achieve demonstrates its advantages in extracting complex market patterns.
satisfying performances.
• Bring in auxiliary knowledge. The auxiliary tasks contains the 6.5. Analysis of alpha performances
valuable information that can enhance the primary task. With
the help of knowledge transfer in MTL, these information can be The Alpha performances of methods indicate the abilities to distin-
transferred into the primary task to help its predictions. guish the stocks according to their profitabilities. Generally, based on
the predicted ranking results, all stocks are divided into 10 levels in
descending orders, where each level contains 10% of stocks. And the
6.3. Effects of TRAN in MTSR profits of each level are measured separately. Top 0% ∼10% represents
the top stocks with the best profitabilities, while Top 90% ∼100%
To achieve better knowledge transfer in the proposed method, represents the bottom (worst) stocks. As illustrated in Fig. 9, the Alpha
TRAN is applied to capture task relations and help the feature learning performances of all methods are estimated in Chinese and American
process. As in the performance comparisons in Tables 2 and 3, it can stock markets. Here are the observations:
be observed that MTSR outperforms MTSR wo TRAN in only Chinese
• Compared with other baselines, MTSR makes more distinct distin-
markets. However, in the backtesting performances in Fig. 6, MTSR
guishments among the stocks in both Chinese and American stock
outperforms MTSR wo TRAN in two markets.
markets, which demonstrates its superiority in Alpha discovery
For a more detailed analysis of the effects of TRAN, the learned task
(stock selection).
weights of TRAN are illustrated in Fig. 8. The task weights indicate the
• MTSR is able to clearly distinguish the top (0% ∼10%) and
contribution proportions of each task in the feature learning towards bottom (90% ∼100%) stocks at the same time, which is usually
target tasks. As we can observe from the illustration, in both Chinese called long-short trading signals. This indicates its effectiveness
and American stock markets, the stock ranking task plays a major role in the two-way trading strategies.
in knowledge transfer, followed by respectively trend prediction task
and excess return prediction task. This observation is consistent with 6.6. Analysis of different trading strategies
the task settings in our method. And the contribution gap between tasks
is relatively more obvious in the Chinese stock market, where the task As presented in Fig. 9, MTSR is able to provide both long and short
weights of stock ranking have reached more than 80%. While in the trading signals. We further evaluate the backtesting performances of
American market, the task weights of stock ranking are around 60%. MTSR with different trading strategies.:

12
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Fig. 9. Alpha performance illustration of all methods. Alpha performance indicates the ability of methods to distinguish the stock Alpha profitabilities. Generally, based on the
predicted ranking results, all stocks are divided into 10 levels in descending orders, where each level contains 10% of stocks. And the profits of each level are measured separately.
Then use the measurements to estimate whether the model is able to distinguish stocks according to their profitabilities.

• Long strategy: Same as the trading strategy in the previous help learn the hardly-trained list-wise ranking task. Additionally, Task
backtesting experiments. Hold the long positions of the top 𝑘 Relation Attention is utilized to capture the dynamic task relations to
stocks and hold for 𝑑 days, after that, clear positions and hold achieve better knowledge transfer between tasks. The experiments con-
new top 𝑘 stocks, and repeat. ducted in the real-world stock datasets demonstrate the effectiveness of
• Short strategy: Hold the short positions of the bottom 𝑘 stocks MTSR and its superiority over several state-of-the-art baseline methods.
and hold for 𝑑 days, after that, clear positions and hold new To our best knowledge, this is the first time to utilize DMTL to address
bottom 𝑘 stocks, and repeat. the training difficulty of list-wise stock ranking. The joint learning
• Long-short strategy: Hold both the long and short positions of framework of MTSR can be applied by the other methods if they want
respectively top 𝑘 and bottom 𝑘 stocks, hold for 𝑑 days, after to perform list-wise stock ranking or multiple stock prediction tasks,
that, clear positions and hold new top 𝑘 and bottom 𝑘 stocks, and improving not only the learning stability, but also the performance.
repeat. In the future work, we aim to discover more inner correlations be-
Holding short positions can be regarded as obtaining the opposite tween different stocks in the ranking prediction, further improve the
revenues of stock returns. In the experiments, we assign 𝑘 and 𝑑 as 10. performance of list-wise stock ranking.
The strategy performances are illustrated in Fig. 10. It can be observed
that MTSR are effective in all these strategies. Additionally, we can
CRediT authorship contribution statement
observe that the short signals in Chinese markets provided by MTSR
are more effective in the backtesting period. These results demonstrate
its effectiveness in further. Tao Ma: Conception and design of study, Experiments and analysis,
Writing – original draft, Revising the manuscript. Ying Tan: Conception
7. Conclusion and design of study, Writing – original draft.

In this paper, a novel DMTL solution is proposed for stock predic-

Declaration of competing interest
tion, which aims to address the training difficulty of list-wise stock
ranking and improve the prediction performance. It utilizes the joint
learning framework of DMTL to learn the list-wise stock ranking with The authors declare that they have no known competing finan-
the enhancements of auxiliary tasks. With DMTL, the easily-trained cial interests or personal relationships that could have appeared to
tasks act as learning guider, providing extra gradient backpropagation, to influence the work reported in this paper.

13
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Fig. 10. Investment performances with different strategies.

Acknowledgements Fischer, T., & Krauss, C. (2018). Deep learning with long short-term memory networks
for financial market predictions. European Journal of Operational Research, 270(2),
654–669.
This work is supported by the National Natural Science Foundation
Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on
of China (Grant No. 62076010), and partially supported by Science and computer vision (pp. 1440–1448).
Technology Innovation 2030 - ‘‘New Generation Artificial Intelligence’’ Gorynin, I., Monfrini, E., & Pieczynski, W. (2017). Pairwise Markov models for stock
Major Project (Grant Nos.: 2018AAA0102301 and 2018AAA0100302). index forecasting. In 25th European signal processing conference, EUSIPCO 2017, Kos,
Greece, August 28 - September 2, 2017 (pp. 2041–2045). IEEE, [Link]
(Ying Tan is the corresponding author.)
23919/EUSIPCO.2017.8081568.
All authors approved the version of the manuscript to be published. Gu, S., Tan, Y., & He, X. (2010). Discriminant analysis via support vectors.
Neurocomputing, 73(10-12), 1669–1675.
References Hashimoto, K., Xiong, C., Tsuruoka, Y., & Socher, R. (2016). A joint many-task model:
Growing a neural network for multiple NLP tasks. arXiv preprint arXiv:1611.01587.
Hu, W., & Tan, Y. (2015). Prototype generation using multiobjective particle swarm
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., et al. (2016). Tensorflow:
optimization for nearest neighbor classification. IEEE transactions on cybernetics,
a system for large-scale machine learning. In OSDI. Vol. 16 (pp. 265–283).
46(12), 2719–2731.
Abdulnabi, A. H., Wang, G., Lu, J., & Jia, K. (2015). Multi-task CNN model for attribute
Jin, Z., Yang, Y., & Liu, Y. (2020). Stock closing price prediction based on sentiment
prediction. IEEE Transactions on Multimedia, 17(11), 1949–1959.
analysis and LSTM. Neural Computing and Applications, 32(13), 9713–9729. http:
Box, G. E., & Pierce, D. A. (1970). Distribution of residual autocorrelations in
//[Link]/10.1007/s00521-019-04504-2.
autoregressive-integrated moving average time series models. Journal of the
Johansen, S. (1991). Estimation and hypothesis testing of cointegration vectors in
American Statistical Association, 65(332), 1509–1526.
Gaussian vector autoregressive models. Econometrica, 1551–1580.
Caruana, R. (1997). Multitask learning. Machine Learning, 28(1), 41–75. Kim, K.-j. (2003). Financial time series forecasting using support vector machines.
Chakraborty, K., Mehrotra, K., Mohan, C. K., & Ranka, S. (1992). Forecasting the Neurocomputing, 55(1–2), 307–319.
behavior of multivariate time series using neural networks. Neural Networks, 5(6), Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv
961–970. preprint arXiv:1412.6980.
Deng, L., Hinton, G., & Kingsbury, B. (2013). New types of deep neural network Li, W., Bao, R., Harimoto, K., Chen, D., Xu, J., & Su, Q. (2020). Modeling the
learning for speech recognition and related applications: An overview. In Acoustics, stock relation with graph network for overnight stock movement prediction. In
speech and signal processing (ICASSP), 2013 IEEE international conference on (pp. C. Bessiere (Ed.), Proceedings of the twenty-ninth international joint conference on
8599–8603). IEEE. artificial intelligence (pp. 4541–4547). [Link], [Link]
Deng, S., Zhang, N., Zhang, W., Chen, J., Pan, J. Z., & Chen, H. (2019). Knowledge- 2020/626.
driven stock trend prediction and explanation via temporal convolutional network. Li, H., Shen, Y., & Zhu, Y. (2018). Stock price prediction using attention-based multi-
In S. Amer-Yahia, M. Mahdian, A. Goel, G. Houben, K. Lerman, J. J. McAuley, R. input LSTM. In J. Zhu, & I. Takeuchi (Eds.), Proceedings of machine learning
A. Baeza-Yates, & L. Zia (Eds.), Companion of the 2019 world wide web conference, research: vol. 95, Proceedings of the 10th Asian conference on machine learning,
WWW 2019, San Francisco, CA, USA, May 13-17, 2019 (pp. 678–685). ACM, ACML 2018, Beijing, China, November 14-16, 2018 (pp. 454–469). PMLR, URL
[Link] [Link]
Ding, Q., Wu, S., Sun, H., Guo, J., & Guo, J. (2020). Hierarchical multi-scale Gaussian Li, C., Song, D., & Tao, D. (2019). Multi-task recurrent neural networks and higher-
transformer for stock movement prediction. In C. Bessiere (Ed.), Proceedings of the order Markov random fields for stock price movement prediction: Multi-task RNN
twenty-ninth international joint conference on artificial intelligence (pp. 4640–4646). and higer-order MRFs for stock price classification. In Proceedings of the 25th
[Link], [Link] ACM SIGKDD international conference on knowledge discovery & data mining (pp.
Duong, L., Cohn, T., Bird, S., & Cook, P. (2015). Low resource dependency parsing: 1141–1151). ACM.
Cross-lingual parameter sharing in a neural network parser. In Proceedings of Liu, L., Zheng, S., & Tan, Y. (2015). S-metric based multi-objective fireworks algorithm.
the 53rd annual meeting of the association for computational linguistics and the 7th In 2015 IEEE Congress on Evolutionary Computation (CEC) (pp. 1257–1264). IEEE.
international joint conference on natural language processing (Volume 2: short papers). Lu, Y., Kumar, A., Zhai, S., Cheng, Y., Javidi, T., & Feris, R. (2017). Fully-adaptive
Vol. 2 (pp. 845–850). feature sharing in multi-task networks with applications in person attribute classifi-
Feng, F., He, X., Wang, X., Luo, C., Liu, Y., & Chua, T. (2019a). Temporal relational cation. In Proceedings of the IEEE conference on computer vision and pattern recognition
ranking for stock prediction. ACM Transactions on Information Systems, 37(2), (pp. 5334–5343).
27:1–27:30. [Link] Luong, M.-T., Le, Q. V., Sutskever, I., Vinyals, O., & Kaiser, L. (2015). Multi-task
Feng, F., He, X., Wang, X., Luo, C., Liu, Y., & Chua, T.-S. (2019b). Temporal relational sequence to sequence learning. arXiv preprint arXiv:1511.06114.
ranking for stock prediction. ACM Transactions on Information Systems (TOIS), 37(2), Ma, T., & Tan, Y. (2020a). Adaptive and dynamic knowledge transfer in multi-task learning
27. with attention networks.

14
T. Ma and Y. Tan Expert Systems With Applications 199 (2022) 116886

Ma, T., & Tan, Y. (2020b). Adaptive and dynamic knowledge transfer in multi- Wang, J., Zhang, Y., Tang, K., Wu, J., & Xiong, Z. (2019). Alphastock: A
task learning with attention networks. In Y. Tan, Y. Shi, & M. Tuba (Eds.), buying-winners-and-selling-losers investment strategy using interpretable deep rein-
Communications in computer and information science: vol. 1234, Data mining and big forcement attention networks. In Proceedings of the 25th ACM SIGKDD international
data - 5th international conference, DMBD 2020, Belgrade, Serbia, July 14-20, 2020, conference on knowledge discovery & data mining (pp. 1900–1908). ACM.
Proceedings (pp. 1–13). Springer, [Link] Xu, K., Zhang, Y., Ye, D., Zhao, P., & Tan, M. (2020). Relation-aware transformer
Ma, T., & Tan, Y. (2020c). Multiple stock time series jointly forecasting with multi- for portfolio policy learning. In C. Bessiere (Ed.), Proceedings of the twenty-ninth
task learning. In 2020 International joint conference on neural networks, IJCNN 2020, international joint conference on artificial intelligence (pp. 4647–4653). [Link],
Glasgow, United Kingdom, July 19-24, 2020 (pp. 1–8). IEEE, [Link] [Link]
1109/IJCNN48605.2020.9207543. Yang, Y., & Hospedales, T. (2016). Deep multi-task representation learning: A tensor
Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., & Chi, E. H. (2018). Modeling task rela- factorisation approach. arXiv preprint arXiv:1605.06391.
tionships in multi-task learning with multi-gate mixture-of-experts. In Proceedings of Yu, Z., Wu, F., Zhang, Y., Tang, S., Shao, J., & Zhuang, Y. (2014). Hashing with list-wise
the 24th ACM SIGKDD international conference on knowledge discovery & data mining learning to rank. In S. Geva, A. Trotman, P. Bruza, C. L. A. Clarke, & K. Järvelin
(pp. 1930–1939). ACM. (Eds.), The 37th international ACM SIGIR conference on research and development in
Misra, I., Shrivastava, A., Gupta, A., & Hebert, M. (2016). Cross-stitch networks for information retrieval, SIGIR ’14, Gold Coast , QLD, Australia - July 06 - 11, 2014 (pp.
multi-task learning. In Proceedings of the IEEE conference on computer vision and 999–1002). ACM, [Link]
pattern recognition (pp. 3994–4003). Zhang, G. P. (2003). Time series forecasting using a hybrid ARIMA and neural network
Pai, P.-F., & Lin, C.-S. (2005). A hybrid ARIMA and support vector machines model in model. Neurocomputing, 50, 159–175.
stock price forecasting. Omega, 33(6), 497–505. Zhang, L., Aggarwal, C. C., & Qi, G. (2017). Stock price prediction via discovering multi-
Preethi, G., & Santhi, B. (2012). Stock market forecasting techniques: A survey. Journal frequency trading patterns. In Proceedings of the 23rd ACM SIGKDD international
of Theoretical and Applied Information Technology, 46(1), 24–30. conference on knowledge discovery and data mining, Halifax, NS, Canada, August 13
Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., & Cottrell, G. W. (2017). A - 17, 2017 (pp. 2141–2149). ACM, [Link]
dual-stage attention-based recurrent neural network for time series prediction. In Zhang, X., & Tan, Y. (2018). Deep stock ranker: A LSTM neural network model for
C. Sierra (Ed.), Proceedings of the twenty-sixth international joint conference on artificial stock selection. In Y. Tan, Y. Shi, & Q. Tang (Eds.), Lecture notes in computer
intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017 (pp. 2627–2633). science: vol. 10943, Data mining and big data - third international conference, DMBD
[Link], [Link] 2018, Shanghai, China, June 17-22, 2018, Proceedings (pp. 614–623). Springer,
Ruder, S. (2017). An overview of multi-task learning in deep neural networks. arXiv [Link]
preprint arXiv:1706.05098. Zhao, J., Du, B., Sun, L., Zhuang, F., Lv, W., & Xiong, H. (2019). Multiple relational
Said, S. E., & Dickey, D. A. (1984). Testing for unit roots in autoregressive-moving attention network for multi-task learning. In A. Teredesai, V. Kumar, Y. Li,
average models of unknown order. Biometrika, 71(3), 599–607. R. Rosales, E. Terzi, & G. Karypis (Eds.), Proceedings of the 25th ACM SIGKDD
Tan, Y., Deng, C., & Ruan, G. (2009). Concentration based feature construction international conference on knowledge discovery & data mining, KDD 2019, Anchorage,
approach for spam detection. In 2009 International Joint Conference on Neural AK, USA, August 4-8, 2019 (pp. 1123–1131). ACM, [Link]
Networks (pp. 3088–3093). IEEE. 3292500.3330861.
Tan, Y., Xia, Y., & Wang, J. (2000). Neural network realization of support vector meth- Zheng, Z., & Tan, Y. (2013). Group explosion strategy for searching multiple targets
ods for pattern classification. 6, In Proceedings of the IEEE-INNS-ENNS International using swarm robotic. In 2013 IEEE Congress on Evolutionary Computation (pp.
Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges 821–828). IEEE.
and Perspectives for the New Millennium (pp. 411–416). IEEE. Zhou, Y., & Tan, Y. (2011). Gpu-based parallel multi-objective particle swarm
optimization. International Journal of Artificial Intelligence, 7(A11), 125–141.

15