INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

Artificial Neural Networks architectures for stock price prediction:

comparisons and applications
Luca Di Persio Oleksandr Honchar
University of Verona University of Verona
Department of Computer Science Department of Computer Science
Strada le Grazie, 15 - Verona Strada le Grazie, 15 - Verona
Italy Italy
[Link]@[Link] [Link]@[Link]

Abstract: We present an Artificial Neural Network (ANN) approach to predict stock market indices, particularly
with respect to the forecast of their trend movements up or down. Exploiting different Neural Networks archi-
tectures, we provide numerical analysis of concrete financial time series. In particular, after a brief résumé of the
existing literature on the subject, we consider the Multi-layer Perceptron (MLP), the Convolutional Neural Net-
works (CNN), and the Long Short-Term Memory (LSTM) recurrent neural networks techniques. We focus on the
importance of choosing the correct input features, along with their preprocessing, for the specific learning algo-
rithm one wants to use. Eventually, we consider the S&P500 historical time series, predicting trend on the basis
of data from the past days, and proposing a novel approach based on combination of wavelets and CNN, which
outperforms the basic neural networks ones. We show, that neural networks are able to predict financial time series
movements even trained only on plain time series data and propose more ways to improve results.

Key–Words: Artificial neural networks, Multi-layer neural network, Convolutional neural network, Long short-
term memory, Recurrent neural network, Deep Learning, Stock markets analysis, Time series analysis, financial
forecasting

1 Introduction timization problems related to the dividend and invest-

ment policy of a firm under debt constraints, resp. to
During recent years, Artificial Neural Networks portfolio liquidation startegies for large portfolio posi-
(ANN) have been interested by a renewed attention tion in a limit-order market, and to [3], and references
by academicians, as well as by practitioners, partic- therein, for models related to the spread of contagion
ularly within the financial arena. The turning point in financial networks. Is it worth to mention that,
being represented by the winning solution for Ima- for what concerns the machine learning approaches,
geNet challenge, in 2012, which was based on deep they are widely applied also to credit scoring, portfo-
convolutional network trained on Graphic Processing lio management, algorithmic trading, see, [22, 24, 10],
Units (GPUs), see [18]. After such an event, ANN respectively. In the present work we are dealing with
approaches to deep learning have been used in lot of the ANN-machine learning approach to the study of
different areas such, e.g., autonomic car driving, robot stock price movements prediction, a theme of increas-
aided surgery, guidance of intelligent drones, software ing relevance in actual financial markets, particularly
development, nuclear physics, and finance. Concern- from the point of view of the so called fast trading
ing the financial framework, different type of mod- management of order books.
els related to the problem of predicting assets prices
movements as well as the behaviour in time of more The paper is structured as follows: in Sec. 2
structured financial instruments, have been consid- we provide a short review of known techniques used
ered. In most of the cases, proposed approaches have for the stock price prediction and mainly based on
been mainly based on statistical models used to anal- classic machine learning algorithms, logistic regres-
yse time series of interest, see, e.g., [12, 13, 14] and sion or support vector machine; in Sec. 3, we con-
references therein, or by exploiting stochastic calculus sider more specifically the Multilayer perceptron, the
techniques, see, e.g., [7, 8], and references therein, or Convolutional neural network, and the Recurrent neu-
by considering fine tuned expansions techniques, as, ral network architectures; then; in Sec. 4, we dis-
e.g., in [11, 9], see also [20], and references therein, cuss input data for algorithms, feature selection and
while we refer to [5, 6], and references therein, for op- preprocessing; eventually, in Sec. 5, we provide

ISSN: 1998-4464 403

 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

the computational results related to the training pro- tors such, e.g., simple and exponential moving aver-
cesses, based on S&P500 time series; in Sec. 6 ages, rate of change, stochastic oscillator, etc. The
we take into consideration optimized selection proce- aforementioned references are all characterized by an
dures for the(hyper)parameters characterizing our ar- accuracy in stock price movement forecasting that
chitectures, focusing the attention on the use of the ranges between 70% and 90%. In what follows, we
Tree-structured Parzen Estimator (sequential model- show that is possible to achieve the same accuracy by
based) optimization scheme; in Sec. 7 we consider mean of neural networks, working with preprocessed
possible alternatives for our the study of the consid- open/close/high/low data, also working with high fre-
ered S&P500 time series from the machine learning quent, intra-day, data.
point of view, namely the Principal Component Anal-
ysis, the feature selection approach and the use of en-
sembles of neural networks; eventually, in Sec. 8, we 3 Artificial neural networks
provide a numerical as well as a graphical interpreta-
tion of the results obtained exploiting the aforemen- 3.1 Multilayer perceptron
tioned approaches, and then commenting them in Sec.
9. Artificial Neural Networks look like electronic models
based on the neural structure of the brain, which ba-
sically learns from experience. The simplest kind of
2 Existing approaches neural network is a single-layer perceptron network,
which consists of a single layer of output nodes, while
The approaches used to forecast future directions of the inputs are fed directly to the outputs via a series of
share market prices are historically splitted into two weights. In this way, it can be considered the sim-
main categories: those that rely on technical analysis, plest kind of feed-forward network. Multilayer neural
and those that rely on fundamental analysis. Techni- network consists of multiple layers of computational
cal analysis uses only historical data such as, e.g., past units, usually interconnected in a feed-forward way.
prices, volume of trading, volatility, etc., while funda- Each neuron in one layer has directed connections to
mental analysis is based on external information like, the neurons of the subsequent layer. In many applica-
e.g., interest rates, prices and returns of other assets, tions the units of these networks apply a sigmoid func-
and other macro/micro-economic parameters. tion as an activation function. The universal approx-
The machine learning approach to the latter prob- imation theorem for neural networks states that every
lem has been declined in several forms, especially dur- continuous function that maps intervals of real num-
ing recent years, also aiming to find an effective way bers to some output interval of real numbers can be
to predict sudden financial crashes as, e.g., the one approximated arbitrarily closely by a multi-layer per-
happened during the 2007-08 biennium. As an exam- ceptron with just one hidden layer. This result holds
ple, good results have been obtained using linear clas- for a wide range of activation functions as in the case
sifiers as the logistic regression one, which has been of sigmoidal functions.
used to predict the Indian Stock market, see [15], or
with respect to the S&P500 index, see [4]. More com-
plicated techniques, as large-margin classifier or Sup- 3.2 Convolutional neural network
port Vector Machine (SVM), was the best choice for Convolutional neural network is a type of feed-
prediction before the rise of neural networks. The lat- forward artificial neural network in which the connec-
ter uses a kernel trick which allows to consider our in- tivity pattern between its neurons is inspired by the
put data as embedded into a higher-dimensional space, organization of the animal visual cortex, whose indi-
where it is linearly separable. Successful applications vidual neurons are arranged in such a way that they
of SVM have been developed, e.g., with respect to respond to overlapping regions tiling the visual field.
the analysis the Korea composite stock prices mar- There are several different theory about how to pre-
ket, see [26], and the NIKKEI 225 index, see [16], cisely define such a model, but all of the various im-
clearly showing how SVM outperforms logistic re- plementations can be loosely described as involving
gression, linear discriminant analysis and even some the following process:
neural network approaches. Another popular tech-
nique based on decision trees, i.e. the random forest ◦ convolve several small filters on the input image;
one, was also applied to similar problems as in, e.g., ◦ subsample this space of filter activations;
[19], where the authors studied the BM&F/BOVESPA
stock market, resulting in a high accuracy obtained ◦ repeat steps 1 and 2 until your left with sufficiently
by using as input features different technical indica- high level features;

ISSN: 1998-4464 404

 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

◦ use a standard a standard MLP to solve a particular

task, using the results features as input.
The main advantage of convolutional neural net-
works is that we use convolutional and downsampling
layers as learnable feature extractor, which allows us
to feed neural network without sophisticated prepro-
cessing, so that useful features will be learnt during
the training. In our case, instead of considering the
standard framework of 2D images, we apply convolu-
tional network to 1D data, namely to (financial) time
series.

3.3 Recurrent neural network

A recurrent neural network (RRN) is any artificial
neural network whose neurons send feedback signals Figure 1: S&P500 index data from 2006 to 2016.
to each other. The idea behind RNNs is to make use
of sequential information. In a traditional neural net-
work we assume that all inputs (and outputs) are in- such renormalization strongly depends on the activa-
dependent of each other. But for many tasks this is tion function of every neuron. A different approach,
not the better idea. In particular, if one wants to pre- see, e.g., [7], is about considering not raw open or
dict the next word in a sentence, then it is better if he close ticks, but calculating return during the day, resp.
knows which words came before it. RNNs are called during the minute, and then using this data as the train-
recurrent because they perform the same task for ev- ing set. In Fig. 2, we have reported the plot of returns
ery element of a sequence, with the output being de- for the S&P500 index.
pended on the previous computations. Another way to
think about RNNs is that they have a memory which
captures information about what has been calculated
so far. It is worth to mention that, eve in RNNs are
theoretically able to treat arbitrarily long sequences
of information, in practice they are limited to looking
back only for few steps.

4 Data processing
In what follows we focus our attention on the S&P500
index, exploiting related data from 1950 to 2016
(16706 data points). Usually stock market data looks
like on Figure 1 and 2, which report the close prices
for every days in the aforementioned time interval
(Fig.1).
Our goal is to predict the movements of the
S&P500 index, exploiting some information from pre- Figure 2: S&P500 index daily returns data.
vious data. Suppose we are going to predict if the
close price of the next day, resp. minute, is larger In our research we use return prices as more rep-
or smaller than the previous one, based on the last resentative data with normalization for stock price
30 days, resp. minutes, of observations. Appropriate movement forecasting problem. In particular, we nor-
time window and prediction horizon should be chosen malized our time series to have zero mean and unit
during hyper parameter optimization stage. First we variance, making use of the sklearn library, see [23].
have to scale or normalize our input data. The input All existing time series have been splitted, accord-
vectors of the training data are normalized in such a ingly to the latter, in train, 80%, resp. in test dataset,
way that all the features have zero-mean, and unitary the remaining 20%; moreover we use 10% of training
variance. Usually data are scaled to be in the range set for hyper parameter optimization. Every element
[-1; 1], or [0; 1]. In the neural network approaches of the train data set is a normalized time series with

ISSN: 1998-4464 405

 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

length of 30. Based on such a subdivision, we want to (ReLU) function as activation function, and, between
predict the transition [1; 0] if price goes up, resp. [0; two hidden layers, one dropout layer is included to
1] if it goes down, next day, resp. minute, according prevent over fitting. The architecture of this model is
with the particular index chosen. shown in Fig. 3.

4.1 Preprocessing for regression problem

A classical approach in time series forecasting is rep-
resented by regression, predicting the exact value in
the future based on given known historical time win-
dow. Neural networks can face this problem as well.
The main difference from the point of view of the clas-
sification architecture, consists in changing the last
affine layer which turns to predict real value. It fol-
lows that the activation function should be a linear
function within the range -inf, +inf, moreover only
one neuron should be present in this layer, instead of
number of classes.
There is an important issue affecting the data
preparation stage before regression. In fact, we need
first to normalize data to be in some fixed range, them,
after the regression procedure ends, we have to restore
obtained results to predict actual value. We can sup-
pose that both the mean and the standard deviation
from historical time window are the same as for the
next value, so that, for training data, we first normal-
ize every time window and then subtract its mean and
divide for its standard deviation, in single prediction Figure 3: MLP architecture.
value. We point out that a common mistake consists
in considering the the mean and the standard deviation
for target as a part of time window, affecting a lot the
final results.

5.2 Convolutional neural network architec-

5 Experimental results ture
In this section we provide the computational results We use the Convolutional neural network architecture
related to the training processes. All NNs were (CNN), as a sequential combination of 1D convolu-
trained using the Keras deep learning framework for tional and max-pooling layers, choosing hyper param-
Python. Every NNs for S&P500 data was trained for eters as follows:
10 epochs, exploiting the Adam optimization algo- ◦ Number of filters = 64;
rithm, see [17]. Computational time was reduced by
using GPU hardware, in particular we made use of a ◦ Pool length = 2;
GTX 860M to speed up tensor multiplication, as well ◦ Subsample length = 1;
as other mathematical routines. In the hyperparame-
ters optimization section, see Sec. 6, we tune all the ◦ Activation function ReLU.
parameters, e.g., the number of hidden layers, neu- We provide experiment results with 1 and 2 hid-
rons, optimization algorithm, etc., with respect to ar- den convolutional layers. The architecture of this
chitectures. model is shown in Fig. 4.

5.1 Multilayer perceptron architecture

We use the Multilayer perceptron architecture (MLP)
with two hidden layers and test different layer sizes
to determine the optimal size from the point of view
of data modelling. We choose the rectified linear unit

ISSN: 1998-4464 406

 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

Figure 5: RNN architecture.

6 Hyperparameters optimization
We need to select the best parameters for our archi-
tectures such as number of neurons, hidden layers,
dropout ratio, optimization algorithm, and the num-
ber of filters in CNNs. It can be done by hyperpa-
rameters search, namely by using the following main
techniques: grid search, random search and the tree-
structured Parzen Estimator.
The Tree-structured Parzen Estimator (TPE), see
[1], is a sequential model-based optimization (SMBO)
approach. SMBO methods sequentially construct
models to approximate the performance of hyper-
parameters based on historical measurements, and
then subsequently choose new hyperparameters to
test based on this model. The TPE approach mod-
Figure 4: CNN architecture. els P (x|y)P (x|y) and P (y)P (y), where x represents
the hyperparameters, while y is the associated qual-
ity score. In particular, P (x|y)P (x|y) is modeled by
5.3 Recurrent neural network architecture transforming the generative process of hyperparame-
As a particular type of Recurrent neural network ar- ters by replacing the distributions of the configuration
chitecture (RNN), we choose the Long Short-Term prior with non-parametric densities.
Memory (LSTM) architecture, with 2 stacked recur- The main steps of the aforementioned procedure
rent layers and a softmax in the end. The number of are the following:
hidden neurons equals 100, while the activation func- ◦ describe the search space, that is hyperparameters
tions are hyperbolic tangent, while the inner activa- and possible values for them;
tions are hard sigmoid functions. The architecture as- ◦ implement the function to minimize, which is noth-
sociated to the latter model is shown in Fig. 5. ing but our neural network model;
◦ optionally, take care of logging values and scores for
5.4 Experimental Wavelet + CNN analysis.
One of the non-traditional approaches for financial The function to minimize takes hyperparameters
time series prediction with NNs is the so called values as input and returns a score, that is a value for
Wavelet-NNs (WNN). Basically, it feature extraction error, since we are minimizing. The latter means that
is based in wavelet transform, taking details part and each time the optimizer decides on which parameter
skipping trend, which are then elaborated exploting a values it likes to check, we train the model and pre-
MLP, see, e.g., in [25]. Since CNNs showed better dict targets for validation set, or we perform a cross-
convergence and accuracy in previous experiments, validation. Then we compute the prediction error and
hence we decided to pass time series preprocessed give it back to the optimizer. After the latter step, it
with Haar wavelets to the 2-layer CNN. again decides which values to check, and the cycle

ISSN: 1998-4464 407

 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

starts over. In particular, we consider the following Value Decomposition (SVD) technique for time se-
set of parameters ries. SVD-type approaches aim at keeping only the
most significant singular vectors to project the data to
a lower dimensional space. PCA is mostly used as a
6.1 The set of parameters for MLP tool in exploratory data analysis and for making pre-
◦ N of hidden layers - choice from [2, 3]; dictive models. The PCA approach implies an eigen-
◦ N of neurons in every layer [64, 126, 250, 512]; value decomposition of a data covariance (or correla-
tion) matrix, or a singular value decomposition of a
◦ Values of dropout rates - uniformly distributed - 0, data matrix, usually after mean centring, and normal-
0.5; izing or using Z-scores. We would like to underline
◦ Optimizing algorithm - choice from Adam, that such an approach is useful to reduce the number
AdaDelta and RMSProp. of variables when using algorithms that do not rely on
time nature of input data, such as feed-forward neu-
After 100 iterations of TPE was chosen model ral networks. In our analysis we have chosen the 16
with two hidden layers, first with 500 neurons, sec- most relevant variables by exploiting the PCA tech-
ond with 250. Dropout values were chosen quite high nique, which allows us to achieve the results reported
- 0.75 and 0.6 for both layers. Best optimizing algo- in subsection. 8.2.3
rithm is Adam.

7.2 Feature selection

6.2 The set of parameters for CNN
Another classical approach in machine learning is the
◦ N of convolution filters - choice from [32, 64]
so called feature selection, which can be used for the
◦ Filter length - choice from [2, 3] selection of feature as well as for dimensionality re-
duction on sample sets, or even to improve the estima-
◦ Pool length - choice from [2, 3]
tors accuracy scores, and to boost their performance
◦ N neurons in fully-connected layer - choice from on very high-dimensional datasets. There are several
[100, 250] variants on how to use it:
◦ Value of dropout rate - uniformly distributed - 0, 1 ◦ Variance threshold: this is a simple baseline ap-
proach to feature selection. It removes all fea-
◦ Optimizing algorithm - choice from Adam,
tures whose variance does not meet some thresh-
AdaDelta and RMSProp
old. By default, it removes all zero-variance fea-
After 100 iterations of TPE, a two hidden layers tures, namely those features that have the same value
model has been selected, with filter length of 2, pool in all samples.
length of 2, and with 64 filters in only one convolu-
tional layer. Dense layer has to have 250 neurons and ◦ Univariate feature selection: such an approach
optimization algorithm has to be AdaDelta. works by selecting the best features based on uni-
variate statistical tests. It can be seen as a prepro-
cessing step to an estimator.
6.3 The set of parameters for RNN
◦ Recursive feature elimination (RFE): this approach
◦ N of cells in every recurrent layer - choice from [50, starts working on a given external estimator that as-
100]; signs weights to features, e.g., the coefficients of a
◦ Optimizing algorithm - choice from Adam, linear model, then the RFE consists in selecting fea-
AdaDelta and RMSProp. tures by recursively considering smaller and smaller
sets of them. First, the estimator is trained on the
After 50 iterations of TPE, a two hidden layers
initial set of features and weights are assigned to
model has been selected, with 100 neurons each, op-
each one of them. Then the features whose abso-
timized by Adam.
lute weights are the smallest, are pruned from the
current set of features. The next step consists in re-
cursively repeat the same procedure on the pruned
7 Further experiments set, until the desired number of features to select is
7.1 Principal components analysis eventually reached.
The Principal Components Analysis (PCA), see, e.g., ◦ A particularly effective procedure is constituted by
[21], can be seen as an algorithm for linear dimen- implementing a feature selector, by exploiting some
sionality reduction. It is mainly based on the Singular trained model. As an example, we can use random

ISSN: 1998-4464 408

 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

trees algorithms which are very robust and precise 8 Results

in classification tasks. In particular such an imple-
mentation allow us to compute feature importances, 8.1 Neural networks for regression
which in turn can be used to discard irrelevant fea- Using the architectures described in previous sections,
tures. we have faced our problem as a regression problem. In
figures 6, 7 and 8, we report a graphical visualization
We have used the last option, choosing important of obtained predictions
features from AdaBoost algorithm, from sklearn li-
brary.

7.3 Ensembles of neural networks

In machine learning, particularly in the creation of
artificial neural networks, ensemble averaging is the
process of creating multiple models and combining
them to produce a desired output, as opposed to creat-
ing just one model. Frequently an ensemble of mod-
els performs better than any individual one, mainly
because the various errors of the models average out.
In particular, we use the following strategy: first
we create a set of experts with low bias and high vari- Figure 6: MLP architecture. Green line - actual value,
ance, and then we average them. Latter approach im-
red line - predicted
plies the creation of a set of experts with varying pa-
rameters which are frequently represented by the ini-
tial synaptic weights. Nevertheless, we would like to
underline that other factors, such as the learning rate,
momentum etc., may be varied as well. Summing up,
we implement the following steps:

◦ generate N experts, each with their own initial val-

ues;

◦ train each expert separately;

◦ combine the experts and average their values.

It is worth to mention that it is also possible to

consider a combination of two or more models in or-
der to create a new one, which performs better than
any of them. In fact, since different models have dif- Figure 7: CNN architecture. Green line - actual value,
ferent weak and strong sides, ad hoc blending proce- red line - predicted
dure of them, may significantly improve the perfor-
mance obtained by each single component. In order Looking at the MLP,CCN and RRN graphs, we
to concretely realize such an approach, the key idea clearly observe that predicted values are behind the
we use is to learn weights of model using some simple actual data, which implies that neural networks can
classifier as, e.g., logistic regression or another neural follow the trend, but cannot predict the exact future
network. values of interest.
We used ensembles of three types: average,
weighted and blending. We chose the best models 8.2 Neural networks for classification
from previous experiments in order to achieve better
results using their ensemble. As blending top classi- 8.2.1 Raw features
fier we have used a regular logistic regression to get In Fig. 15 we have displayed the results related to the
the weights of simple predictors. training of different architectures on daily returns. It

ISSN: 1998-4464 409

 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

Figure 11: CNN loss function within 10 epochs

Figure 8: RNN architecture. Green line - actual value,

red line - predicted

is easy to see that, from this perspective, CNNs are the

best choice for modelling financial time series. Such
a result suggests us to use CNN as basic predictor.
Plots of accuracy and losses changing during training,
are reported in figures from 9 to 14. In particular such Figure 12: CNN accuracy within 10 epochs
graphs show that, due to the stochastic nature of data
we are working with, the training procedure performs
slowly and messy.

Figure 13: RNN loss function within 10 epochs

Figure 9: MLP loss function within 10 epochs

Figure 14: RNN accuracy within 10 epochs

8.2.2 Wavelet-CNN
After training CNN on wavelet decomposed data, with
respect to the details of a signal, we obtain: MSE =
Figure 10: MLP accuracy within 10 epochs 0.24, with an accuracy value equal to 0.55, hence ob-
taining a slight increase of performance if compared

ISSN: 1998-4464 410

 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

MSE Accuracy MSE Accuracy

MLP 0.26 0.521 Average ensemble 0.23 0.558
CNN 0.2491 0.536 Hand-weighted ensemble 0.23 0.56
RNN 0.2498 0.522 Blended ensemble 0.226 0.569

Figure 15: MSE and accuracy after training different Figure 16: MSE and accuracy after training different
architectures ensembles

with previous results based on raw data. 9 Conclusions

In this work different artificial neural network ap-
proaches, namely MLP, CNN, and RNN, have been
8.2.3 PCA selected features applied to the forecasting of stock market price move-
ments. We compared results trained on a daily ba-
We would like to point out that the application of CNN sis, for the S&P500 index, showing, in particular, that
to PCA selected features, turns to produce poor re- convolutional neural networks (CNN) can model fi-
sults. In fact, by implementing such an approach we nancial time series better then all the other considered
lose the time-based structure of data. Then we trained architectures.
MLP on PCA-transformed data, achieving MSE = We also implemented a novel Wavelet + CNN al-
0.26, with an accuracy value of 0.51, hence a worse gorithm which outperforms other neural network ap-
result, even if compared with raw trained networks. proaches. In particular, it shows that feature prepro-
Latter fact is not surprising, because in this case time cessing is one of the most crucial parts in stock price
series is ruined by having removed some data points. forecasting.
We would like to underline that we achieved
much better results using ensembles of neural net-
8.2.4 Tree-based selected features works. It is worth to mention that such an approach
has showed its power particularly when dealing with
We can train CNN on data from tree-selected features, large-scale classification tasks, such, e.g., Netflix cus-
in order to choose CNN as a classifier, because ran- tomer classification, see [2].
dom trees chose last data points as significant data for Our ongoing researches aim at treating more ex-
classification, hence without corrupt the time series tended financial time series, and their features, in or-
structure. Applying this method we achieve: MSE = der to improve the training performance as well as re-
0.256, with an accuracy value of 0.525, hence a result lated accuracy, for the particular type of neural net-
at least comparable to the one obtained by the MLP works considered. Techincal indicators, moving aver-
approach. ages and stochastic oscillators, will be also taken into
account to increase accuracy of prediction.

8.2.5 Ensembles of neural networks

Exploiting the results reported along previous sec-
tions, we have chosen the CNN trained on raw fea-
tures model, the W-CNN model and the RNN model,
to create an ensemble which performs better than its
single components. We have reported in Fig. 16 the
results obtained applying average, hand-weighted en-
semble and blending.
As hand weights, we have empirically chosen 0.3
for CNN, 0.25 for RNN and 0.45 for W-CNN, as the
most accurate predictor.
After learning blending with logistic regression
we realized that it was hypothesis - learned (and nor-
malized) weights were 0.32 for CNN, 0.19 for RNN
and 0.49 for W-CNN, which shows us, that result of
W-CNN is the most important for an ensemble.

ISSN: 1998-4464 411

 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

References: [14] Di Persio, L. and Perin, I., An ambit stochastic

[1] [Link] et al. Algorithms for Hyper-Parameter approach to pricing electricity forward contracts:
Optimization NIPS 2015 The case of the German Energy Market (2015)
Journal of Probability and Statistics.
[2] James Bennett, The Netflix Prize Proceedings of
[15] A. Dutta, G. Bandopadhyay and S. Sengupta,
KDD Cup and Workshop, 2007
Prediction of Stock Performance in the Indian
Stock Market Using Logistic Regression, Inter-
[3] Benazzoli, C., Di Persio, L. , Default contagion national Journal of Business and Information,
in financial networks International Journal of 2012
Mathematics and Computers in Simulation, 10, [16] Wei Huang, Yoshiteru Nakamori, Shou-Yang
pp. 112-117, 2016 Wang, Forecasting stock market movement di-
[4] S.-[Link], Genetic Algorithms and Genetic rection with support vector machine, Computers
Programming in Computational Finance, and Operations Research, archive Volume 32, Is-
Springer, 2002 sue 10, 2005, pp. 2513–2522
[5] Chevalier, E., Vath, V.L., Scotti, S., An opti- [17] D. P. Kingma, J. Lei Ba, Adam: A method for
mal dividend and investment control problem stochastic optimization, 3rd Int. Conf. for Learn-
under debt constraints SIAM Journal on Finan- ing Representations, San Diego, 2015
cial Mathematics, 4 (1), pp. 297-326, 2013 [18] A. Krizhevsky, I. Sutskever and G. E. Hin-
[6] Chevalier, E., Vath, V.L., Scotti, S., Roch, A. ton, ImageNet Classification with Deep Convo-
Optimal execution cost for liquidation through a lutional Neural Networks, NIPS 2012, Nevada,
limit order market International Journal of The- 2012
oretical and Applied Finance, 2016 [19] Marcelo S. Lauretto, B. C. Silva and P. M.
[7] Cordoni, F. and Di Persio, L., Backward stochas- Andrade, Evaluation of a Supervised Learn-
tic differential equations approach to hedging, ing Approach for Stock Market Operations,
option pricing, and insurance problems (2014) arXiv:1301.4944[[Link]], 2013
International Journal of Stochastic Analysis, [20] Marinelli, C., Di Persio and L., Ziglio, G., Ap-
2014. proximation and convergence of solutions to
[8] Cordoni, F. and Di Persio, L., Invariant measure semilinear stochastic evolution equations with
for the Vasicek interest rate model in the Heath- jumps (2013) Journal of Functional Analysis,
Jarrow-Morton-Musiela framework (2015) Infi- 264 (12), pp. 2784-2816.
nite Dimensional Analysis, Quantum Probabil- [21] Pearson, K. On Lines and Planes of Closest Fit to
ity and Related Topics, 18 (3). Systems of Points in Space. Philosophical Mag-
azine 2 ,pp. 559572. , 1901
[9] Cordoni, F. and Di Persio, L., First order cor-
rection for the characteristic function of a multi- [22] Rodan, A., Faris, H. Credit risk evaluation us-
dimensional and multiscale stochastic volatility ing cycle reservoir neural networks with support
model (2014) International Journal of Pure and vector machines readout Lecture Notes in Com-
Applied Mathematics, 93 (5), pp. 741-752. puter Science (including subseries Lecture Notes
in Artificial Intelligence and Lecture Notes in
[10] Cont, R., Kukanov, A. Optimal order placement Bioinformatics) , 9621, pp. 595-604, 2016
in limit order markets Quantitative Finance, pp.
1-19, 2016 [23] Pedregosa, F., Varoquaux, G., Gramfort, A.,
Michel, V., Thirion, B., Grisel, O., Blondel,
[11] Di Persio, L., Pellegrini, G. and Bonollo, M., M., Prettenhofer, P., Weiss, R., Dubourg, V.,
Polynomial chaos expansion approach to interest Vanderplas, J., Passos, A., Cournapeau, D.,
rate models, (2015) Journal of Probability and Brucher, M., Perrot, M. and Duchesnay, E.,
Statistics. Scikit-learn: Machine Learning in Python, Jour-
[12] Di Persio, L. and Frigo, M., Gibbs sampling ap- nal of Machine Learning Research Volume 12,
proach to regime switching analysis of financial 2011, pp.2825–2830
time series (2016) Journal of Computational and [24] Novak, G. Veluscek, D. Prediction of stock price
Applied Mathematics, 300, pp. 43-55. movement based on daily high prices Quantita-
[13] Di Persio, L. and Frigo, M., Maximum like- tive Finance, 16 (5), pp. 793-826, 2016
lihood approach to markov switching models [25] Chong Tan, Financial Time Series Forecasting
(2015) WSEAS Transactions on Business and Using Improved Wavelet Neural Network, Mas-
Economics, 12, pp. 239-242. ter Thesis, University of Aarhus, 2009

ISSN: 1998-4464 412

 INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING Volume 10, 2016

[26] Y. Wang and In-Chan Choi, Market Index and

Stock Price Direction Prediction using Machine
Learning Techniques: An empirical study on the
KOSPI and HSI, Technical report, 2013
[27] [Link] and Yu. A Kuperin, Using
Recurrent Neural Networks To Forecasting of
Forex, arXiv:cond-mat/0304469 [[Link]-
nn], 2003

ISSN: 1998-4464 413