Kernel-based Machine Learning Methods for

High-Frequency Financial Time Series Analysis

by

Andreas Koukorinis

First Supervisor: Professor Guido Germano

Second Supervisor: Professor Gareth Peters

A thesis submitted for the part fulfillment of the requirements for the degree of Doctor
of Philosophy (Ph.D) in

COMPUTER SCIENCE

Faculty of Engineering
Department of Computer Science
University College London
Date of submission: Monday 16th September, 2024
2

Declaration

I, Andreas Koukorinis, confirm that the work presented in this thesis is my own. Where

information has been derived from other sources, I confirm that this has been indicated in the
work.
Abstract

This thesis investigates machine learning and statistical methodologies for modelling time
series to classify and examine market dynamics and transitions. The research is motivated
by the impact of algorithmic trading as observed in trade-and-quote data (henceforth, TAQ),
which introduces challenges in modelling non-stationarities with irregular information flow.
Across four chapters, innovative applications for time series modelling are discussed. The
focus is on short-term market regime classification and statistical comparison testing on
multifractal detrendend fluctuation analysis (MFDFA) statistics, emphasizing the development
of adaptive and interpretable frameworks.

1. Methodology
Chapter 3 establishes the methodological background of the research topics.
We begin with the representation problem, then move to kernel methods and learning
paradigms, and finally hidden Markov models. We also discuss algorithms for construct-
ing generative feature spaces, multiple kernel learning approaches, and kernel target
alignment. Maximum mean discrepancy is introduced for measuring distribution dis-
tances in a reproducing kernel Hilbert space (RKHS). Finally, we discuss long-memory
and persistence concepts, including appropriate statistical methods for time series data.
This theoretical framework supports the models developed in subsequent case studies
in Chapters 5 & 6. Appendix C provides a technical treatment of kernels and related
topics.

2. Data and statistical transformations

This chapter introduces TAQ data and techniques to analyze such datasets. It presents
information clock, an alternative to calendar-based sampling, which samples data

3
4

based on market activity variables. An algorithm is introduced to create appropriate

class labels to classify imbalanced data. Visualization techniques such as t-SNE and
RadViz are used to demonstrate how kernelization improves classification accuracy.
This chapter lays the foundation for subsequent data-driven approaches, highlighting
their adaptability to evolving market dynamics and the robustness of our methodology.

3. Generative-discriminative machine learning models for regime classification

Chapter 5 presents a novel hybrid learning framework. The proposed HMM-SVM-MKL
(hidden Markov kernel machine) methodology uses model-based generative feature
embeddings to extract informative features from TAQ data, capturing market dynamics
and improving predictive accuracy. This is demonstrated by classifying six intraday
regimes using data from 40 FTSE100 stocks.

4. Statistical investigation in memory and persistence in interest rate futures con-

tracts
Chapter 6, examines the memory and persistence properties of limit order book-related
quantities (from TAQ data) of interest rate futures contracts. Through the use of in-
formation clocks, multifractal techniques, and distributional testing, we investigate the
statistical properties of these quantities, revealing significant variations in memory
exponents, multifractal spectra, and power-law tails.

Contributions to science This research contributes to the scientific research of high-

frequency financial data modeling and its practical applications, encompassing algorithmic
trading strategies, risk management frameworks, and the analysis of microstructure phenom-
ena. We also present the methdology for the data and its motivating literature in Chapter 4.

1. Model-based generative feature embeddings

Model-based generative feature embeddings have been developed to extract features
from TAQ time series data. These embeddings are designed to capture the critical
features of the data, which are essential to incorporate the patterns and behaviors of the
market microstructure. This approach improves the interpretability and effectiveness
 5

of machine learning models applied to financial time series, enabling more accurate
predictions and better insights into market dynamics.

2. Hybrid machine learning approach

A hybrid machine learning approach, HMM-SVM-MKL, is proposed, integrating hidden
Markov models and kernel learning (single or multiple). This combination leverages the
strengths of generative models in capturing temporal dynamics and the discriminative
power of kernel-based methods. This framework effectively captures complex market
behaviors and improves predictive accuracy, demonstrating its empirical success in
classifying six distinct intraday regimes using high-frequency data from 40 FTSE100
stocks.

3. Framework for statistical investigation on long memory and persistence

Chapter 6 examines the memory and persistence properties of microstructure-linked
variables in interest rate futures contracts. A combination of multifractal detrended
fluctuation analysis and kernel two-sample testing tests the distribution of statistical
characteristics of limit order book data variables. The results provide insights into long
memmory and persistence, enabling Bayesian inferences about the dominant mecha-
nisms governing asset dynamics across trading timeframes. The use of information
clocks ensures that the true dynamics of market behavior is captured. This contribution
advances the field by offering a structured approach to analyze order book-linked
MFDFA data.
Impact Statement

Electronic financial markets are complex, dynamic ecosystems shaped by trading events,
generating large amounts of time-series data that capture their intricate workings. Modelling
such data presents unique challenges, including non-stationarity, noise, and multidimensional
dependencies, which traditional approaches often struggle to address effectively. Algorithmic
traders must navigate these market complexities while trying to design adaptive strategies.
The constant evolution of markets demands a level of adaptability that conventional methods
do not provide. Using machine learning techniques, this thesis will illustrate how innovation
translation effectively addresses market complexity and facilitates the design of adaptive
strategies capable of evolving alongside the markets within which they operate. Machine
learning techniques address these modelling challenges by providing methodologies de-
signed to detect complex patterns in data, making few assumptions about functional forms.
This enables more flexible modelling tailored to the properties of financial time series data.

1. Methodology
This chapter discusses methods for effective innovation translation. We present es-
tablished kernel techniques appropriate for high-frequency financial data modelling.
We also introduce concepts such as hybrid learning and generative embeddings. By
applying machine learning techniques to financial data, we showshow innovation can
lead to more interpretable and effective models for market regime classification and
microstructure analysis.

2. Data and transformations

Across both of the empirical studies, we propose the use of information clocks instead of
calendar-time sampling to better capture market activity. Clocks based on trade counts,

7
8

volume, or dollar value capture the discrete asynchronous arrival of new data, [1].
These techniques enable the adaptation of trading strategies to information arrival
rates and provide insight into market microstructure dynamics across different trading
horizons.

3. Automated labeling
We also introduce a novel automated method for generating class labels in high-
frequency data, which is crucial for supervised learning in algorithmic trading. This
methodology facilitates the application of supervised learning algorithms, while enabling
the systematic identification and classification of salient market events and potential
trading signals, thus enhancing the efficacy of quantitative trading strategies.

4. Regime classification
The proposed hybrid framework combines probabilistic models with discriminative
learning techniques, using model-based generative feature embeddings to extract
informative features from high-frequency financial data. This approach captures market
dynamics and improves predictive accuracy and interpretability, as demonstrated in
classifying six intraday regimes using data from 40 FTSE100 stocks. We consider
microstructure models as learning models [2–4]. We aim to advance the modelling of
market dynamics and improve the classification of market regimes.

5. Long memory and persistence

The second empirical study examines the memory and persistence properties of
microstructure quantities derived from the TAQ data (of interest rate futures). The
methodology developed leverages multifractal techniques and kernel two-sample test-
ing distributional testing, in a two-stage approach. Significant variations in memory
exponents, multifractal spectra, and power-law tails, are identified which highlight
complex structures in the microstructure of the market [5–7].

6. Market inefficiencies and risk management

The developed methodologies offer techniques for identifying market inefficiencies
 9

and anomalous behaviour. Research contributes to better risk management in high-

frequency trading strategies. This work offers empirical insights for decision-making
in dynamic financial environments. Identifying inefficiencies, volatility clustering, and
regime shifts showcases the method’s strengths.

7. Contributions
We advance theoretical understanding of high-frequency financial data modelling and
offer practical applications for algorithmic trading, risk management, and market mi-
crostructure analysis. The thesis provides a framework for addressing the complexities
of the modern market, potentially improving market access, reducing volatility, and
improving surveillance. Variants of the methodologies has been applied at JP Morgan,
where the candidate was managing the Systematic Market Making group globally.

8. Broader impact
The methodologies developed can inform regulatory policies, aid in market surveillance
efforts, and contribute to more stable and efficient financial markets. From a policy
perspective, this framework can help oversee systemic vulnerabilities arising from
memory effects. The interdisciplinary nature of this work, integrating machine learning,
statistical inference, and computational finance, may inspire further research across
these fields.
10

Acknowledgements

I am deeply grateful to my supervisors, Professor Guido Germano and Professor Gareth

Peters, for their invaluable guidance and support throughout my Ph.D. studies. Professor
Peters played a particularly critical role, providing constant inspiration and insight that greatly
enriched my research. Both their technical and editorial advice was essential for bringing
this research to its current form. I am especially grateful for their excellent advice, direction
and suggestions, the enthusiasm they expressed for my work and the flexibility and freedom
I was provided to explore new avenues of research. I could not have completed this work
without their commitment and encouragement over the past 10 years.
The first case study was a collaborative effort, with the core algorithm designed together
with Professors Peters and Germano and Dr. Konstantinos Manolarakis. Professor Germano
initially proposed the idea of using an algorithm for automated label creation, which became
a key contribution. Dr. Manolarakis was instrumental in implementing the software used in
the study.
Most importantly, I dedicate this thesis to my beloved wife Litha Pepas and my daughter
Aemilia. Their unwavering love and support makes everything worthwhile.
Contents

List of Figures 17

List of Tables 19

List of Algorithms 21

Notation 25

1 Introduction 27
1.1 Key motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.1.1 Key ideas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.1.2 Statement of objectives . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.2 Scope and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
1.3 Significance and contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 34
1.4 Thesis organisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2 Background and literature review 39

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.2 Key concepts in high-frequency financial data . . . . . . . . . . . . . . . . . . 40
2.2.1 Market microstructure . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.2.2 Periodicity and information efficiency . . . . . . . . . . . . . . . . . . . 41
2.2.3 Information clocks and event time . . . . . . . . . . . . . . . . . . . . 42
2.3 Machine learning in algorithmic trading . . . . . . . . . . . . . . . . . . . . . 45
2.3.1 Kernel methods and support vector machines . . . . . . . . . . . . . . 45
2.3.2 Fisher kernels and generative embeddings . . . . . . . . . . . . . . . 46
2.3.3 Multiple kernel learning . . . . . . . . . . . . . . . . . . . . . . . . . . 48

11
12 Contents

2.4 Selected topics in time series analysis in markets . . . . . . . . . . . . . . . . 50

2.4.1 Long memory and persistence . . . . . . . . . . . . . . . . . . . . . . 50
2.5 Advanced statistical techniques . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.5.1 Kernel two-sample testing . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.5.2 Multifractal detrended fluctuation analysis . . . . . . . . . . . . . . . . 52
2.6 Statistical properties of financial markets . . . . . . . . . . . . . . . . . . . . 55
2.6.1 Scaling laws and power-law distributions . . . . . . . . . . . . . . . . . 56
2.6.2 Volatility clustering and correlation structures . . . . . . . . . . . . . . 56
2.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

3 Methodology 59
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2 Brief overview of kernel methods . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.3 Supervised and generative learning . . . . . . . . . . . . . . . . . . . . . . . 64
3.4 Overview of hidden Markov models . . . . . . . . . . . . . . . . . . . . . . . 67
3.5 Feature spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.5.1 Fisher score algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.5.2 State space embedding algorithm . . . . . . . . . . . . . . . . . . . . 69
3.5.3 Transition space embedding algorithm . . . . . . . . . . . . . . . . . . 70
3.5.4 Emission space embedding algorithm . . . . . . . . . . . . . . . . . . 71
3.6 Multiple kernel learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.6.1 Traditional multi-kernel learning formulation . . . . . . . . . . . . . . . 73
3.6.2 Kernel target alignment . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.7 Overview of kernel two-sample testing . . . . . . . . . . . . . . . . . . . . . . 80
3.7.1 Two-sample testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.8 Long memory and multifractal long memory . . . . . . . . . . . . . . . . . . . 82
3.8.1 Slow hyperbolic decay in the context of long memory . . . . . . . . . . 83
3.8.2 Statistical techniques for modeling long memory . . . . . . . . . . . . 86
3.9 Overview of MFDFA and MF-DCCA . . . . . . . . . . . . . . . . . . . . . . . 87
Contents 13

3.9.1 Algorithms for MFDFA and MFDCCA . . . . . . . . . . . . . . . . . . . 88

3.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

4 Data and their statistical transformations 93

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.3 Information clock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.3.1 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.4 Automated class label construction . . . . . . . . . . . . . . . . . . . . . . . . 101
4.5 Data and methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
4.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.6.1 Parameter values for labels . . . . . . . . . . . . . . . . . . . . . . . . 112
4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

5 Generative-discriminative machine learning models for regime classification 117

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.2 Background and hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.3 Modelling methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.3.1 Market structure captured by hidden Markov models . . . . . . . . . . 125
5.4 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.4.1 Examining feature embedding stability and empirical characteristics . . 133
5.4.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
5.5 Experimental results for kernel classifiers . . . . . . . . . . . . . . . . . . . . 136
5.5.1 Benchmark experiments . . . . . . . . . . . . . . . . . . . . . . . . . 137
5.5.2 Experiments comparing kernel classification algorithms . . . . . . . . 140
5.5.3 Comparing across different labels/regimes . . . . . . . . . . . . . . . . 144
5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

6 Statistical investigation in memory and persistence in interest rate futures

contracts 149
14 Contents

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

6.2 Background and hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

6.3 Modelling methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

6.3.1 Interpreting the MFDFA statistics . . . . . . . . . . . . . . . . . . . . . 156

6.3.2 KTST Inference procedure . . . . . . . . . . . . . . . . . . . . . . . . 158

6.4 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

6.5 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

6.5.1 Experimental design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

6.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

6.6.1 Experiments on the mass exponent τ and singularity spectrum of the

microprice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

6.6.2 Orderflow imbalance . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

6.6.3 Multifractal detrended cross-correlation experiments . . . . . . . . . . 172

6.6.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

6.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

7 Conclusions 179

7.1 Summary of key ideas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

7.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

7.3 Further work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

7.3.1 Hybrid deep learning models . . . . . . . . . . . . . . . . . . . . . . . 182

7.3.2 Connecting the multifractal spectrum and information theoretic mea-

sures of market efficiency . . . . . . . . . . . . . . . . . . . . . . . . . 183

7.3.3 Non-stationary extensions of kernel two-sample tests for analyzing

regime changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

A Limit order book 185

A.1 Limit order book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

A.2 Mathematical description of a limit order book . . . . . . . . . . . . . . . . . . 186

Contents 15

A.2.1 Mathematical framework for limit order book dynamics . . . . . . . . . 190

B Hidden Markov models 191

B.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
B.2 Hidden Markov models for sequential data modeling . . . . . . . . . . . . . . 191
B.2.1 Problem definition and notation . . . . . . . . . . . . . . . . . . . . . . 192
B.2.2 Key HMM inference problems . . . . . . . . . . . . . . . . . . . . . . . 192
B.3 Tractable algorithms for inference, decoding, and learning . . . . . . . . . . . 194
B.3.1 Advances in HMM training . . . . . . . . . . . . . . . . . . . . . . . . 198

C Kernels 203
C.1 Background on kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
C.1.1 Reproducing kernel Hilbert spaces . . . . . . . . . . . . . . . . . . . . 203
C.1.2 The Representer theorem and its implications for kernel machines . . . 205
C.1.3 Formulation of the learning problem within the context of RKHS . . . . 206
C.2 Learning, testing and expressing structure with kernels . . . . . . . . . . . . . 207
C.3 Expressing structure through kernels . . . . . . . . . . . . . . . . . . . . . . . 208
C.3.1 Families of kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
C.3.2 Stationary versus non-stationary kernels . . . . . . . . . . . . . . . . . 212
C.3.3 Isotropic versus anisotropic kernels . . . . . . . . . . . . . . . . . . . 213
C.4 Overview of kernel learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
C.5 Deep multiple kernel learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
C.5.1 Preliminaries and notation for DMKL . . . . . . . . . . . . . . . . . . . 222
C.6 Inference procedures for time-series data: kernel two-sample testing . . . . . 227
C.6.1 Time series data embedding to kernel RKHS space . . . . . . . . . . . 229

Bibliography 237
List of Figures

1.1 Themes and topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.1 Discriminative versus generative learning . . . . . . . . . . . . . . . . . . . . 66

3.2 Express high-level properties of functions with kernels . . . . . . . . . . . . . 73
3.3 Example of multi- kernel learning . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.4 Long memory versus short memory processes . . . . . . . . . . . . . . . . . 87

4.1 Labelling mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

4.2 Generic pipeline for data sampling . . . . . . . . . . . . . . . . . . . . . . . . 107
4.3 Feature correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.4 Shapiro rankings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.5 t-SNE samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.6 CPG.L RadViz for Label 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.7 REL.L RadViz for Label 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.8 KGF.L RadViz for Label 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.9 KGF.L RadViz for Label 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.10 Threshold-based price labeling mechanism . . . . . . . . . . . . . . . . . . . 114
4.11 Distribution of proportion of negative versus positive classes for Label 5. . . . 115
4.12 Distribution of proportion of negative versus positive classes for Label 2. . . . 115
4.13 Distribution of proportion of negative versus positive classes for Label 3. . . . 115
4.14 Distribution of proportion of negative versus positive classes for Label 1. . . . 115

5.1 Workflow of the experimental evaluation . . . . . . . . . . . . . . . . . . . . . 124

5.2 Duration vs. traded price . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.3 Hidden Markov model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

17
18 List of Figures

5.4 Distribution of median correlation coefficients across different trading symbols 133
5.5 Simulation study for features . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.6 Classification performance for six labels across all methods . . . . . . . . . . 143

6.1 Two-stage experimental process . . . . . . . . . . . . . . . . . . . . . . . . . 155

6.2 Correlation heatmaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.3 Data sampling process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
6.4 Two-sample test visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
6.5 Autocorrelation decay comparison . . . . . . . . . . . . . . . . . . . . . . . . 168
6.6 Multifractal spectrum of trading Dynamics . . . . . . . . . . . . . . . . . . . . 170
6.7 OBI probability density and CDF plots . . . . . . . . . . . . . . . . . . . . . . 171
6.8 Volume impacts cross-correlation distributions . . . . . . . . . . . . . . . . . . 172

A.1 Visualization of a Limit Order Book (LOB) . . . . . . . . . . . . . . . . . . . . 186

A.2 State of LOB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
A.3 LOB Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

B.1 Hidden Markon model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

C.1 Universal kernel property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

C.2 RBF kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
C.3 Stationary versus non-stationary kernels . . . . . . . . . . . . . . . . . . . . . 213
C.4 Isotropic versus anisotropic kernels. . . . . . . . . . . . . . . . . . . . . . . . 215
C.5 Examples of stationary isotropic kernels . . . . . . . . . . . . . . . . . . . . . 216
C.6 SVM criterion: the margin between classes . . . . . . . . . . . . . . . . . . . 219
C.7 Witness function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
C.8 Test statistic MMD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
List of Tables

0.1 Matrix and vector notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

0.2 Kernel and stochastic process notations . . . . . . . . . . . . . . . . . . . . . 26
0.3 Notation for limit order book concepts . . . . . . . . . . . . . . . . . . . . . . 26

3.1 Comparison of multi- kernel learning methods. . . . . . . . . . . . . . . . . . 80

4.1 Parameter settings for labels . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

5.1 Summary of descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . 130

5.2 Descriptive statistics for FTSE stock data. . . . . . . . . . . . . . . . . . . . . 131
5.3 Effectiveness of generative embeddings . . . . . . . . . . . . . . . . . . . . . 138
5.4 Performance metrics for all classification models . . . . . . . . . . . . . . . . 143
5.5 MKL Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
5.6 KTA results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

6.1 Global trading dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

6.2 KTST metrics analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
6.3 Hypothesis tests results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
6.4 OBI Power law analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.5 Volume impact on microprice persistence . . . . . . . . . . . . . . . . . . . . 173
6.6 Hurst of cross-correlation of microprice and volatility . . . . . . . . . . . . . . 174
6.7 Cross-correlation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

19
List of Algorithms

1 Fisher score algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

2 State space embedding algorithm . . . . . . . . . . . . . . . . . . . . . . . . 70
3 Transition-state embedding algorithm . . . . . . . . . . . . . . . . . . . . . . 71
4 Emission space embedding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5 MFDFA algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6 MFDCCA algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7 KTST optimizing parameters and testing. . . . . . . . . . . . . . . . . . . . . 160
8 Forward-Backward algorithm for HMM state estimation . . . . . . . . . . . . . 196
9 Viterbi algorithm for most likely sequence estimation . . . . . . . . . . . . . . 197
10 Baum-Welch algorithm for HMM parameter estimation . . . . . . . . . . . . . 198

21
Publications and Conference Talks

Parts of the work in this thesis have been submitted or published as follows.

• Andreas Koukorinis, Gareth Peters, Guido Germano, "Hybrid generative-discriminative

machine learning models for forecasting financial time series", Proceedings of the 8th
International Conference on Computational and Financial Econometrics, Dec 2014.

• T. Spooner, Andreas Koukorinis, J. Fearnley and R. Savani "Market Making via

Reinforcement Learning," in Proceedings of the 17th International Conference on
Autonomous Agents and MultiAgent Systems, Stockholm, Sweden, 2018, pp. 434-442.
doi: 10.5555/3237383.3237450.

• Andreas Koukorinis, Gareth Peters, Guido Germano, "Hybrid Generative-Discriminative

Machine Learning Models for High-Frequency Financial Time Series", submitted to
Methodology and Computing in Applied Probability, doi: 10.21203/[Link]-4497881/v1.

• Andreas Koukorinis, Gareth Peters, Guido Germano, "Examining Statistical Anoma-

lies and Persistence in Interest Rate Futures: A statistical investigation on the mesoscale
using kernel two-sample testing", submitted to Journal of Finance and Data Science.

• Yu, Jialin, Andreas Koukorinis, Nicolo Colombo, Yuchen Zhu, and Ricardo Silva.
"Structured Learning of Compositional Sequential Interventions", arXiv:2406.05745,
doi: 10.48550/arXiv.2406.05745, submitted to NeurIPS 2024.

In addition, parts of this work have been presented in the following conferences and
workshops:

23
24 List of Algorithms

• June 2019 – Workshop on Economic Science with Heterogeneous Interacting Agents,

"A comparative study of stylised facts across various markets: explaining common
behaviours with latent variable models", Talk.

• 2014 – 8th International Conference of Computational and Financial Econometrics, "Hy-

brid generative-discriminative (HMM-SVM) machine-learning models for the forecasting
of multivariate financial time series", Talk.

• 2014 – LSOLDM: Large-Scale Online Learning and Decision Making Workshop, "Hybrid
learning for algorithmic trading.", Talk.
Notation

Table 0.1: Matrix and vector notations

Symbol Description
x Scalar number.
x Vector.
X Matrix.
xi Element i of vector x.
xj Row j of matrix X.
XT Transpose of matrix X.
X−1 Inverse of matrix X.
|X| Determinant of matrix X.
⟨x, y⟩ Inner product of vectors x and y.
Each element of vector x is less than the corresponding element in the vector
x<y
y.
vec(A) Vector form of matrix A.
∇α f (α) Gradient of a scalar function f (α) with respect to α.

25
26 List of Algorithms

Table 0.2: Kernel and stochastic process notations

Symbol Description
Squared-exponential kernel, also known as the radial-basis function (RBF)
SE
kernel, the Gaussian kernel, or the exponentiated quadratic.
Per Periodic kernel.
Lin Linear kernel.
Changepoint kernel, σ(x, x′ ) = σ(x)σ(x′ ), where σ(x) is a sigmoidal function
σ
such as the logistic function.
ka + kb Addition of kernels, shorthand for ka (x, x′ ) + kb (x, x′ ).
ka × kb Multiplication of kernels, shorthand for ka (x, x′ ) × kb (x, x′ ).
k(X, X) Gram matrix, whose i, j th element is k(xi , xj ).
K Shorthand for the Gram matrix k(X, X).
f (X) Vector of function values, whose ith element is given by f (xi ).
{Zi }Ti=0 A discrete-time stochastic process.
Z Set of states: {ζ1 , . . . , ζK }.
Z Stochastic process representation.
O Observation process in the Hidden Markov Model.
A Transition probability matrix.
Aj,k Transition probability from state ζk to ζj .
π Distribution of the initial state.
log Natural logarithm (base e).
P Density function of the complete data.
POi |Zi =ζ Marginal density of the observation data.
PTi |Zi =ζ Marginal distribution of state durations.
Θ Set of all parameters that define a discrete state H(S)MM.

Table 0.3: Notation for limit order book concepts

Symbol Description
pbid
t Bid price at time t
pask
t Ask price at time t
q bid (p, t) Volume available on bid side at price level pbid
t and time t
q ask (p, t) Volume available on ask side at price level pask
t and time t
mbidt Market order volume arriving on bid side at time t
maskt Market order volume arriving on ask side at time t
δq bid (p, t, t + ∆t) Change in bid volume at price pbid
t due to new limit orders in [t, t + ∆t]
δq ask (p, t, t + ∆t) Change in ask volume at price pask
t due to new limit orders in [t, t+∆t]
Change in bid volume at price pbid
t due to executions against market
ϵq bid (p, t + ∆t)
orders
ϵq ask (p, t + ∆t) Change in ask volume at price pask
t due to executions against market
Chapter 1

Introduction

This chapter introduces the key motivations for the research topic, focussing on applying
machine learning techniques to high-frequency financial time series data. We outline the
challenges inherent in TAQ (trades-and-quotes) data, including non-stationarities and
latent microstructure effects. The research objectives, centered on developing adaptive
and interpretable models, are presented alongside the scope and limitations of the thesis.
Key approaches are, such as, kernel methods, hidden Markov models, multiple kernel
learning, kernel two sample testing and multifractal detrendend fluctuation analysis, are
introduced. The chapter highlights the significant contributions to industry and academic
research, particularly in regime classification and long-memory analysis of financial data.
Finally, the thesis structure and the experiments conducted in subsequent chapters are
presented.

27
28 1.1. Key motivation

1.1 Key motivation

This thesis explores the application of kernel methods to high-frequency financial data
(specifically, trade-and-quote data (TAQ)) in the context of algorithmic trading, addressing
the complexities of modern markets and some of the limitations of traditional approaches.
Such data pose significant challenges because of their volume, dimensionality, and complex
microstructure dynamics. TAQ data often exhibit features such as heavy-tailed distributions,
volatility clustering, and long-range dependencies, which traditional econometric models
struggle to capture [8, 9].

Additionally, irregular information arrival creates stochasticity, requiring statistical analysis

to reveal market dynamics and evaluate the effect of trading activity [10, 11]. Furthermore,
Hasbrouck [12] and Gould et al. [13] highlight that market inefficiencies and microstructure
effects require better analytical techniques. The primary goal is to develop a modelling frame-
work that captures the empirical features of TAQ data and temporary market inefficiencies,
while addressing these challenges. The framework provides insights and develops a set of
scalable, theoretically grounded models that explain key stylized facts.

To address the challenges, the thesis employs kernel methods, which provide a powerful
nonparametric framework for modeling probability densities and capturing nonlinear rela-
tionships in high-dimensional spaces. As emphasized by Gretton et al. [14] kernel methods
offer computational efficiency and flexibility, making them well suited for large-scale learning
problems. Additionally, we use information or event based clocks, as suggested by the work
of Griffin and Oomen [15] and Velasco-Fuentes and Ng [1], better capture the irregular data
arrival process. We aim to constructing accurate models that improve our understanding
of markets. Otherwise, any subsequent reasoning, prediction, or strategy design would be
flawed.
1. Introduction 29

1.1.1 Key ideas

A cross-disciplinary set of ideas is synthesised by leveraging concepts from limit order

book modelling, statistics and machine learning. The primary motivation is to develop
methodologies for modeling and forecasting that address the challenges discussed earlier.

Unique elements of high frequency financial time series data A key motivation is to gain
insights into and exploit microstructure dynamics that lead to market inefficiencies. However,
this requires capturing dependencies between observations that are only milliseconds apart.
The data frequently demonstrates distributional anomalies and non-stationarities arising
from fluctuating volatility, as well as regime shifts. Sudden price jumps and activity spikes
are common. Heterogeneous information arrival due to asynchronous trading leads to
stochasticity in the timing of observations. Careful statistical analysis can reveal patterns
related to market impact, liquidity, risk, algorithmic trading dynamics.

When modeling static data (single observations), empirical approaches are often used to
identify which features to model [16]. For sequence data, however, empirical feature selection
is difficult to implement since the range of potential sequence features/dependencies is large.
A small number of standard models — such as hidden Markov models (HMMs)—are typically
used. Unfortunately, for many applications, the assumptions about the conditional inde-
pendence of these models are incorrect, potentially reducing the accuracy of classification.
Bicego et al. [17], demonstrate and address limitations of standard HMM-based clustering for
sequences by proposing a similarity-based approach, implying that standard HMMs may not
fully capture complex sequence dependencies or features.

Imbalanced data sets A key challenge in classification of time series data, especially in the
context algorithmic trading, is the phenomenon of unbalanced labels (unbalanced learning).
Often, in data sets that represent events like financial fraud, market crashes, or rare health
anomalies, the occurrences of the event of interest (often termed the target label or positive
class) are far outnumbered by nonevents (the negative class). This imbalance introduces a
30 1.1. Key motivation

significant modeling challenge, and we discuss it in Section 4.4.

When subjected to such imbalanced datasets, standard machine learning algorithms,
which are typically optimized for accuracy, may tend to overwhelmingly favor the majority
class. This is because a model that naively predicts the majority class for all observations can
still achieve a misleadingly high accuracy rate. The repercussions of this bias can be severe:
the model might consistently overlook the rare, yet crucial, positive instances. In practical
terms, this could mean missing out on detecting fraudulent financial transactions, overlooking
critical market opportunities, or failing to identify rare health anomalies in real-time patient
monitoring.
The class boundary learned by these conventional algorithms tends to be severely skewed
toward the majority class. Consequently, while the false positive rate might be low, the false
negative rate can increase.
In some particular domains (like finance or healthcare), where the cost of a false negative
(like missing a fraud or health anomaly) can be substantially higher than a false positive,
such models can lead to grave implications. Addressing this challenge requires a departure
from traditional modeling approaches. Techniques like oversampling the minority class,
undersampling the majority class, or leveraging synthetic data generation can help balance
the scales. Beyond data-level approaches, algorithmic adaptations, such as cost-sensitive
learning, where higher penalties are imposed for misclassifying the minority class, can
recalibrate model biases. Saeed et al. [18] demonstrate that SVM with MKL consistently
outperforms single kernel SVMs on imbalanced datasets. By deploying an algorithmic
approach to counteract the effect of imbalanced data [19], we propose a methodology that
is better equipped for trade classification, even in scenarios where the data distribution is
skewed.

Event-based sampling Another important consideration was using an information- or

event-based clock, as an alternative to traditional calendar time for sampling market data.
Indexing observations based on the arrival of new information aligns better with the natural
time scale of the underlying process. Both of the empirical studies, in Chapters 6 and
1. Introduction 31

5 incorporate event-based sampling. Models that incorporate event-based sampling can

characterize non-normally distributed returns, volatility clustering, and discrete jumps more
accurately [1, 20].
These anomalies arise due to the asynchronous and heterogeneous arrival of information
in markets [21] and discrete trading driven by order flow imbalances [22, 23]. Particularly, the
trade clock accounts for the irregular and discontinuous nature of volatility across different
time scales [24].
In this thesis, we leverage information clocks to reveal empirical dependencies, long
memory, fractality, and non-normal distributional forms in TAQ returns. This offer a unifying
framework to analyze anomalies and non-normalities in the data while respecting market
microstructure.

Computational benefits of MKL From a computational perspective, multiple kernel learn-

ing (MKL) efficiently combines numerous kernels, without explicit feature mapping, thus
reducing computational costs and memory requirements. The approach combines weak
kernels into an optimally weighted ensemble for the learning task. The approach is highly
scalable, enabling the combination of very large number of kernels with minimal additional
resources.
This thesis also holds practical advantages over deep neural networks for financial
applications. Although deep learning can model highly non-linear relationships, it requires
vast data volumes and extensive hyperparameter tuning. The proposed techniques focus on
specialized model architectures designed for temporal data rather than sheer model capacity.
This emphasizes model interpretability and transparency alongside predictive accuracy.
The data and computational requirements are realistic for real-time deployment in live
trading environments. Both, of these case studies advance the development of machine
learning-driven methodologies capturing market dynamics, latent states, long-range depen-
dencies, and any distributional abnormalities that may arise. In addition to the domain of
finance, the contributions may inform research across other disciplines working with temporal
data.
32 1.1. Key motivation

1.1.2 Statement of objectives

This research will have two main research questions.

1. We develop generative state-space model-based kernels, such as the Fisher kernel,

to capture the dynamic properties of high-frequency financial data. This approach
integrates generative probabilistic models ( e.g. hidden Markov models), with discrimi-
native learning through multiple kernel learning, creating a hybrid learning framework.
Also, in the first empirical case study, we will address the challenge of unbalanced
classification in times-series within the context of algorithmic trading.

2. The second research question investigates memory, persistence, and distributional

characteristics of order-book derived summary variables at various temporal scales.
We use multifractal detrended fluctuation analysis (MFDFA) as the data-transormation
mechanism for features, on data indexed by information clocks. By employing clocks
indexed to event activity, we aim to uncover empirical dependencies in the data.

Figure 1.1: Mapping of relationship of themes and topics to chapters.

1. Introduction 33

Objectives

The research aims to address the challenges posed by the high complexity and variability the
data, by developing adaptive and interpretable models that can capture nonlinear relationships
and regime shifts in TAQ data. By integrating methods from machine learning, computational
finance, and time-series analysis, this research aims to develop models that can interpret the
market microstructure and its impact on financial data. The primary set of objectives are:

1. Develop machine learning models that are adaptabale and interpretable for the analysis
of TAQ data.

2. Create a modelling framework that captures non-linear relationships and regime shifts
inherent in financial time series. Financial time series data often display non-stationary
characteristics and distributional anomalies, such as heavy-tailed distributions and
volatility clustering. These properties emerge from fluctuating market conditions and
external influences.

3. Address the high complexity and variability in financial data through advanced modeling
techniques.

4. Understand and model the stochastic behaviour and statistical properties of financial
data across different time scales.

5. Provide insights into market dynamics through careful consideration of varying time-
scales in financial data analysis.

1.2 Scope and limitations

This thesis primarily focuses on data from a selection of UK equities traded on the London
Stock Exchange and a selection of interest rate futures contracts traded globally. We
utilize trades-and-quotes data, enabling the study of intraday trading patterns and market
microstructure effects (sampled at various event clocks).
34 1.3. Significance and contributions

While this approach offers valuable insights, it also presents computational challenges
and potential overfitting risks, which are acknowledged and addressed throughout the work.
As discussed earlier our research will center on two main experimental research questions.
For the hybrid modeling framework, we limit our validation to demonstrating its efficacy in
regime classification and its potential for improving trade direction prediction. While other
applications could be explored, they are considered out of scope for this thesis.

The specifications for our models, particularly in the multiple kernel learning (MKL)
approach, are necessarily restricted to a manageable number of configurations. We have
chosen models that are sufficiently general and adaptable, focusing on linear combinations
of kernels in our MKL implementation. More involved kernel combinations or alternative MKL
formulations could be explored in future work. In our MFDFA/KTST study, we concentrate
on detecting long-range dependencies and multifractal properties in TAQ derived statistics.
While the aim is market inefficiency detection and long memory identification, other potential
applications are left for future research.

1.3 Significance and contributions

This research contributes to the literature in several ways:

1. Hybrid learning framework

The integration of generative probabilistic embeddings and multiple kernel learning is a
key contribution. Generative models like hidden Markov models (HMMs) capture the
latent states and dynamics of market microstructure but lack predictive power. This
limitation is addressed by deriving generative embeddings from HMMs, which are used
as features for discriminative learning [25]. Multiple kernel learning extends the frame-
work by combining kernels adapted to these generative embeddings. This approach
produces interpretable features that connect generative unsupervised learning with
discriminative supervised learning, facilitating better classification.

2. MFDFA of market microstructure variables

1. Introduction 35

In Chapter 6, the research addresses key gaps in the study of market microstructure by
introducing a mesoscale timeframe, bridging high-frequency and long-term dynamics.

3. Statistical framework for long memory

A statistical framework is developed to evaluate distributions of long-term memory and
persistence variables.

The proposed methodologies offer practical implications for professionals dealing with TAQ
data by enabling precise modeling of discontinuities, causality, and volatility dynamics while
preserving market microstructure. These contributions, both theoretical and practical, are
relevant not only to finance but also to other fields working with high-frequency temporal data,
facilitating new discoveries in these areas.

Research experiments

The following experiments are organised by the chapter they appear:

Chapter 4: Automated label construction The chapter presents a method employing

dynamic empirical quantile estimates, which is essential for trading data. Processed trade
data are used to calculate returns and inter-trade durations. Labels are generated by
comparing current prices with smoothed series using thresholds for significant deviations.
This adaptive approach captures evolving market conditions, improving the robustness of the
hybrid HMM-SVM-MKL model for practical trading applications.

Chapter 4: Feature visualisation The study uses various feature analysis techniques to
evaluate the inputs of the classification model. It employs multiple label construction methods
and feature ranking algorithms to assess the significance of the features. visualisations
revealed significant insights into feature clustering, separability, and structure, confirming
their effectiveness in capturing financial market data complexities.

Chapter 5: Synthetic data studies for generative embeddings We use synthetic data
studies to evaluate the efficacy of HMM-based generative model features. These experiments
36 1.3. Significance and contributions

show asymptotic convergence of generative embedding features, indicating reliable capture

of underlying data distribution over short timescales.

Chapter 5: HMM-SVM-MKL comparative results This study compares the multiple

kernel learning (MKL) model with generative embeddings, logistic regression, and feed-
forward neural networks. MKL integrates multiple kernels to capture nonlinear decision
boundaries, outperforming single-kernel methods and other benchmarks. It showed superior
performance in regime classification and price prediction, with a lower Hamming loss and
higher classification metrics. The ability of MKL to integrate multiple kernels provided a
robust framework to capture complex patterns in financial data, offering improved predictive
accuracy and trading signal reliability compared to traditional models.

Chapter 6: Kernel two-sample testing on MFDFA variables. This study employs kernel
two-sample testing on multifractal detrended fluctuation analysis variables to investigate
long-range dependence dynamics in financial time series. MFDFA is used to evaluate
scaling behaviours and persistence characteristics of microprice returns and microstructure
data. The kernel two-sample test is then applied to compare distributions of the MFDFA-
derived variables across different datasets, identifying divergences in their long-term memory
properties. The findings reveal statistical irregularities and variations in long-memory features
among futures contracts, suggesting that distinct scaling behaviours are influenced by the
patterns of information arrival.

Chapter 6: Kernel two-sample testing on MFDCCA variables The second set of experi-
ments applies multifractal detrended cross-correlation analysis to study cross-correlations
among microstructure variables. Kernel two-sample testing is also used to assess the sta-
tistical significance and temporal evolution of these cross-correlations. The findings reveal
significant variations in the structures of cross-correlation between trading sessions and
sampling methods, indicating complex and persistent interdependencies. This approach
provides insights into hidden dependencies and long-range memory dynamics, enhancing
1. Introduction 37

the understanding of market efficiency and order flow behaviour under diverse conditions,
while highlighting the importance of multifractal analysis in financial time series.

1.4 Thesis organisation

A literature review that is presented in Chapter 2. The background sections provide an

overview of the relevant literature, while the chapters delve deeper into specific aspects.
The literature directly relevant to each experiment is examined in detail within the respective
experimental studies. This structure ensures the understanding of the current state of the
literature and discusses relevant research in the context of each experiment.
Chapter 3, presents the methodological foundations for the remaining chapters. It covers
topics such as kernel methods, hidden Markov models, multiple kernel learning, kernel
two-sample testing, and multifractal analysis techniques. Additionally, it addresses the
representation problem, supervised and generative learning, hybrid approaches, genera-
tive embeddings, kernel target alignment, long memory concepts, MFDFA and MFDCCA
algorithms, power-law fitting, computational complexity, and feature space analysis. It is
accompanied by Appendices B and C.
Chapter 4, discusses the data and statistical transformation techniques used in the thesis,
including methods for automated class label construction, information clock sampling, and
feature visualisation techniques like t-SNE and RadViz, to analyze the data. Appendix A
accompanies the chapter.
Chapter 5, proposes the application of a hybrid learning framework, based on HMM and
SVM/MKL, for the classification of the high frequency data regime. Chapter 6 combines
ideas from previous chapters, applying multifractal analysis and distributional testing to study
long memory in interest rate futures contracts microstructure variables, revealing distinct
scaling behaviours. Chapter 7 concludes and presents directions for future research.
Chapter 2

Background and literature review

This literature review chapter provides an examination of key research that motivates
the models introduced in later chapters. The content is grounded in both methods
and evidence from the existing literature and is self-contained, allowing it to serve
as a comprehensive reference. However, readers already familiar with the material
may choose to proceed directly to the next chapter. The topics covered include key
microstructure topics, generative-discriminative models in machine learning, multifractal
detrended fluctuation analysis (MFDFA) in finance, and kernel two-sample testing.

39
40 2.1. Introduction

2.1 Introduction

We review key literature on which the models developed in this thesis are based. This
review begins with an overview of key concepts in high-frequency TAQ data in Section 2.2
and topics related to the microstructure of markets. An examination of kernel machines,
generative embeddings and hybrid models follows in Section 2.3, highlighting their potential
in algorithmic trading and TAQ data analysis.

Kernel two-sample testing is introduced as the tool for comparing samples of high-
dimensional data in Section 2.5.1. The concept of the information clock, which forms the
basis for constructing the experimental data sets, is explored in Section 2.2.3. Long memory
in financial time series is examined in Section 2.4.1, with a particular focus on multifractal
detrended fluctuation analysis (MFDFA) techniques. Finally, we review the stylised empirical
facts in financial markets, considering their relevance to high-frequency trading and automated
market making 2.6.

This chapter provides context for both of the approaches and the modeling presented in
subsequent chapters. By synthesizing insights from diverse fields such as market microstruc-
ture [26], kernel methods [14, 27, 28], and multifractal analysis [29], we lay the groundwork
for the research contributions.

2.2 Key concepts in high-frequency financial data

This section discusses select key topics in high-frequency finance, such as market microstruc-
ture, intraday seasonality, information efficiency, and limit order book dynamics, and their
impact of algorithmic trading. Microstructure dynamics informs trading strategies and can
shape the modeling of empirical phenomena. These dynamics affect price discovery, fluctuate
throughout the trading day, and ultimately shape market efficiency and periodicity.
2. Background and literature review 41

2.2.1 Market microstructure

Market microstructure examines how trading mechanisms, particularly the interplay of trades
and orders, affect price formation and market efficiency. This research is primarily based on
trade and quote (TAQ) data and requires an understanding of market structure topics. These
concepts are important for interpreting TAQ data and the implications for price discovery,
liquidity, and efficiency. Dallamico et al. [30], discussed how hidden liquidity levels reveal and
contribute to the observable and unobservable aspects of trading. This partly motivates the
idea that market microstructure can be modelled as a learning model.
Beltran-Lopez et al., show that TAQ data are crucial to measuring trade information
content and the liquidity dynamics of limit order books [31]. Bouchaud, Farmer, and Lillo [26],
explore how markets slowly incorporate large orders into prices. Using high-frequency data
similar to TAQ, they demonstrate that order flow is a long-memory process, challenging
traditional models. Their work reveals that the market impact is nonlinear and history-
dependent, significantly affecting liquidity dynamics and informing trading strategies across
different time scales.
In our work, we challenge the hypothesis that immediate price adjustments follow new
information thus considering order flow as a long-memory process. Overall, as TAQ data
reflects a degree of information about order flows and price formation it justifies its use in this
research to examine market dynamics.

2.2.2 Periodicity and information efficiency

Market periodicity invariance and the degree of information efficiency exhibit interlinked
behaviours across time intervals, indicating multifractality. This non-linear behaviour motivates
our investigation MFDFA statistics of order book summary statistics in Chapter 6. Andersen
et al.[32], investigate how this type of data captures time-varying volatility patterns and
discuss the influence of microstructure effects on intraday price dynamics.
The non-invariant nature of intraday periodicity leads to varying distributions of nor-
malised returns on an intra-day basis. We expand this reasoning, by testing for differences
42 2.2. Key concepts in high-frequency financial data

MFDFA statistics in of micro- price returns across different sampling clocks and reach similar
conclusions.

In [33], the authors investigate the presence of high-frequency periodicity in foreign

exchange markets, which is related to the work in Chapter 6. Using data from 25 currencies,
they discover significant subminute periodicity, which has an effect on liquidity and price
discovery. This periodicity is likely caused by algorithmic participation, particularly from
agency execution algorithms. These types of executions have an impact on market quality as
they can have an effect and influence on price formation and market efficiency.

In related work, Tseng [34] explores the concept of market efficiency in a continuous-time
semi-martingale framework, which is suitable for high-frequency data analysis. It shows that
prices in the interdealer foreign exchange market became more informationally efficient after
a reduction in tick size. This may be attributed to the increased market-making activity by
high-frequency traders.

Easley et al.[35], explore the implications of asymmetric information on asset returns

through a private information factor (PIN). Empirical analysis reveals the effectiveness of
the PIN factor in differentiating cross-sectional returns compared to established benchmarks
such as Fama-French factors. The findings suggest that the risk of toxicity, as captured by
the PIN factor, is an important driver of exchange-traded asset returns.

The findings on information efficiency and the impact of private information align with the
choice of kernel two-sample testing to investigate market inefficiencies ; see Chapter 6.

These studies underscore the importance of the techniques and data transformation
methods introduced later in Chapter 4 for capturing market dynamics.

2.2.3 Information clocks and event time

Studying various phenomena using sampling frequencies that are more natural to the modern
microstructure of financial markets is an essential to our work. Following [36], the concept
of information event will refer to arrival of new information as a quantitative measure of
information arrival based on increments of the information time process. Thus, we can
2. Background and literature review 43

construct a homogeneous time series by sampling at some predefined frequency and then
using an interpolation technique [37].

This concept is not novel, so we adhere to preexisting definitions; see [38–42]. The novelty
in our work is to characterise and discuss the more prevalent stylised facts in information
time. The stochastic clock is supposed to capture the information time, that is, we wish to
sample the market based on the arrival of new information. Therefore, the change from the
micro-to mesoscale by dividing the trading day into buckets used for aggregation creates an
event-by-event summary of a limit order book based on trading activity and the arrival of new
information. Hence, we can classify time series data as homogeneous or in-homogeneous
based on the regularity of their arrival rate in calendar time [1]. For example, raw price data
that arrive at random times is inhomogeneous. A transformation into a homogeneous time
series occurs by sampling at a specific frequency using a different process [1, 43].

The goal in Chapter 6 is to find meso-scopic descriptors of the information clock. Through-
out the thesis, the terms information clock/time or event clock/time are used interchangeably.

The focus is on a statistical description of prices as a function of the information revealed

in the market, whether from the arrival of a trade or the cumulative traded volume being high
enough (and above some threshold) to reveal new information. This premise reflects similar
intuition as past work by Mandelbrot [44], Easley et al. [3], Knuth [45], O’Hara [46] and others.

Easley et al. [47], pioneered modeling information diffusion as a stochastic arrival process
governed by an underlying "information clock". Early empirical work like [48] estimated the
scaling of returns. The authors study the distribution of share volume, which exhibits a
power-law decay, and time correlations in share volume demonstrate long-range persistence.
These results indicate a potential connection between share volume and information clocks,
where the accumulation of trading activity, rather than fixed time intervals, measures the
passage of time.

Theoretical foundation of information clocks can be found in [49], where it is shown that
by changing the probability measure or changing the clock, asset price processes can be
expressed as martingales. Geman [49] showed that changing probability measures or clocks
44 2.2. Key concepts in high-frequency financial data

can express prices as martingales.

Of particular relevance for us is the work of Velasco Fuentes et al. [1], as it investigates
the use of stochastic time changes to restore normality of asset returns. The authors consider
the total number of transactions or trading volume as a stochastic clock. They find that
conditioning only on the number of de-meaned trades does not always restore normality in
asset returns, but using a nonlinear function as the stochastic clock improves the results.
The relationship between market activity and normal returns is examined using parametric
models, and conditioning returns on the number of trades or trading volume is observed to
reduce volatility.

The analysis is based on future tick data from the FTSE-100 index, revealing that normality
is restored only in the less volatile subperiod. The results suggest that the relationship
between economic time, the number of transactions, and trading volume may be more
involved during turbulent market conditions. The restoration of normality in asset returns
enables the use of continuous-time analytical tools and provides a clear definition of economic
time.

Similarly, Holder et al.[50], examine the impact of the active trades and the time between
trades for the treasury market and find that subsequent price and sign of trades are linked to
the duration between trades and how that is measured.

In related work, Huth and Abergel [51], investigate multivariate subordination in financial
markets, potentially offering insights into time-varying relationships between assets. Also
Hardimana et al. [52] explicitly modeled information arrival as a Hawkes self-exciting process,
finding empirical evidence of two linked clock regimes. In Chapter 6, we link the idea of
mesoscale to the concept of the information clock.

In [53], a volume clock is used to construct a measure for toxicity in order flow in the US
futures market, they assert the benefits of their predictor for toxicity induced volatility in that
market. The crucial finding here is that during the week before the 2010 flash crash, there
was large imbalance in the submissions of the market orders, a large number of information
trades were occuring, and the VPIN was also extremely unstable.
2. Background and literature review 45

2.3 Machine learning in algorithmic trading

The success of kernel methods in signal processing and machine learning can be attributed
to their ability to handle nonlinear, high-dimensional data effectively [27]. Similarly, Henrique
et al. (2019) [54] review various machine learning techniques for financial market prediction,
emphasizing the importance of feature selection methods such as principal component
analysis (PCA) and genetic algorithms (GA) before applying machine learning models such
as SVMs.

These preprocessing steps do improve predictive performance by reducing dimensionality

and improving the quality of input features. Kernel machines, particularly support vector
machines (SVMs), have become prominent in the classification and prediction of high-
frequency financial data [55–59].

2.3.1 Kernel methods and support vector machines

Numerous studies have applied SVMs to tasks like forecasting stock price movements and
market direction using high-frequency data; for example [55–59]. The nonlinear modeling
capabilities of SVMs allow them to capture patterns in noisy financial time series [60].

Thissen et al. [61] compared SVM with other methods for time series prediction, highlight-
ing the importance of kernel choice and parameter selection. Their work underscores the
need for careful model tuning when applying kernel machines to this type of data. Kercheval
et al.[62], uses multiclass support vector machines to model high-frequency limit order book
dynamics, generating forecasts of midprice movements and spread crossings using NASDAQ
data.

While showcasing the potential for short-term prediction and profitable trading applica-
tions, the research is subject to certain limitations. The analysis is confined to a narrow
stock sample, presumes market stationarity, and does not benchmark against alternative
approaches. In addition, their approach can overlook complex nonlinear relationships and
external influences. However, this work sets an important foundation for the application of the
46 2.3. Machine learning in algorithmic trading

chosen techniques and motivates Chapter 5. We address some of these limitations in the
chapter.

Although the use of single kernel approaches [63–66] provided the groundwork, multi-
kernel approaches align more closely with the intricacies of financial data and have shown
success in similar applications [62, 67].

2.3.2 Fisher kernels and generative embeddings

In the thesis, we explore the development and application of hidden Markov models (HMMs)
to generate embeddings from limit order books using TAQ data. Building on the concept
of varying intraday distributions, generative embeddings (derived from HMMs) reduce high-
dimensional market data into interpretable features, potentially improving classification accu-
racy and capturing underlying drivers of trading patterns.

The work of Cartea et al.[68–70] on algorithmic trading strategies using similar models to
dynamically learn from market-wide price dynamics provides the primary motivation. Their
work accounts for influential versus non-influential trades, self-exciting order arrival processes,
and temporary versus permanent price impacts. The research demonstrates how to estimate
model parameters from tick data and use the models to develop optimal trading strategies.

Importantly, their work explains how different components of the models relate to ob-
servable limit order book features like spread, depth, and order imbalance. This provides a
practical and theoretical foundation for the decision to extract informative embeddings from
TAQ data using an HMM to model the microstructure dynamics. The hybrid classification
method can leverage these embeddings along with other data sources to predict relevant
outcomes for trading or risk management.

Azencott et al. [71], focuses on real-time market microstructure analysis, leveraging kernel
methods for transaction cost analysis (TCA). They highlight the necessity of automated feature
selection and influence analysis to identify significant factors affecting trading performance.
This approach allows for real-time adjustments in trading strategies, thereby improving
execution efficiency.
2. Background and literature review 47

Generative models can capture the underlying structure of the data and learn the relation-
ships between different variables [72]. Order book embeddings improve classification [73].
Our proposed learning approach in Chapter 5, combines the benefits of generative and
discriminative models and is similar to Lasserre et al. [72].
The work of El-Yaniv and Pidan [74], explored the use of selective prediction models
based on hidden Markov models for the prediction of financial trends.
They introduced the concept of selective HMMs, which abstain from making predictions
when confidence is low. This approach effectively generates embeddings that can be
leveraged to enhance the performance of kernel machines in financial data analysis. This
parallels both the work of Halloran and Rocke [75], who used generative model gradients to
improve peptide identification in tandem mass spectrometry via Fisher kernels. We leverage
the same concepts in Chapter 5.
Visser et al.[76], focused on computing confidence intervals for HMM parameters using
likelihood profiling, bootstrapping, and the finite-differences approximation to the Hessian.
Their work provides a foundation for generating reliable HMM embeddings, which can be
crucial for kernel machines requiring high-confidence feature representations.
This thesis avoids the need for manual feature engineering and builds upon aforemen-
tioned work on extracting information from TAQ data [68, 69, 76]. We extract descriptive
features from a generative model in a manner similar to [77, 78] and use different kernel
classifiers to improve predictive accuracy motivated by the works of Lasserre et al.[72, 79],
Valstart et al.[80], and Fletcher et al.[58, 73].
Brodersen et al. [81], produce DCM-based embeddings that enable model comparison
and physiological interpretation. Bicego et al.[78], show that HMM-based embeddings capture
temporal dynamics and outperform conventional methods in seismic signal classification.

Hybrid models in trading There has been multiple approaches in joining HMM with
discriminative learning methods, in particular support vector machines:
A sequential approach has be used as the technique for combining discriminative and
generative learning; [72]. This involves first generating a set of representative features using
48 2.3. Machine learning in algorithmic trading

a generative model, and then applying discriminative techniques to this feature set. One of
the central concepts in this area is the Fisher Kernel, which is explained in detail.

In [82] support vector machines are used, utilizing dynamic time-warping kernels to
overcome issues around scaling and modeling variable-length time series that SVMs face.
They demonstrate outperformance over static kernels and naive forecasts on futures data.
Key limitations include limited coverage of assets, the lack of comparative analysis with
other machine learning approaches, and potential issues with overfitting due to extensive
parameter tuning.

In recent work, Azim [83] demonstrates that deep generative features, when used with
Fisher kernels, can match or exceed the performance of complex deep models. In [84], the
authors develop a hybrid model combining an HMM-trained Fisher kernel with SVM for FX
data analysis. They also employ Bayes point machines and a DC algorithm for learning HMM
parameters, resulting in improved performance.

We follow some concepts from Montana et al.[85] in this thesis. They propose an algo-
rithmic trading approach using incremental support vector regression experts and ensemble
learning to adapt to non-stationary market environments, outperforming individual models in
risk-adjusted returns.

2.3.3 Multiple kernel learning

The reasons for focusing on multiple kernel learning (MKL) methods are threefold: the
MKL approach is essentially a dynamic ensemble method that constructs a mixture kernel,
thereby encoding complementary information. Secondly, it is known that MKL methods are
appropriate for small but wide datasets [86, 87], as is the case with our high-dimensional
generative embeddings extracted from the HMM.

The MKL approach involves the integration of different kernels, such as polynomials
and RBF helps MKL to capture nonlinear decision boundaries even with limited data, and
kernel-based regularization makes it less likely to overfit [88]. Finally, MKL identifies the
optimal combination of information sources encoded across kernels. This aligns well with our
2. Background and literature review 49

objective of finding the right balance between the generative embedding features [89–91].

In [92] and [93, 93], financially motivated kernels are applied in a multiple-kernel frame-
work. This thesis is significantly influenced by the use order book data and Fisher kernels
in these studies. Fletcher et al.[67], employ multiple kernel kearning to emulate the signal
combination process of technical trading rules, while leveraging SVMs to predict currency
movements across different time horizons.

Imbalanced data A key challenge in this thesis is the imbalanced learning problem. Mar-
ket regimes changes or other events of interest often occur infrequently, resulting in an
imbalanced dataset where certain states or conditions are significantly underrepresented
compared to others that are more common.

An implicit goal is to develop a classifier that identifies the minority class correctly.

This issue becomes particularly critical in datasets where the occurrence of significant
events, such as market crashes or sudden shifts in asset prices, is rare compared to the
overwhelming number of non-events.

Addressing this challenge requires a multifaceted approach. The research by Saeed and
Ong [18], partly motivates our choice of using a multi-kernel learning framework.

Also we leverage Lopez et al. [19], who suggest that algorithmic modifications optimize
classifiers for imbalance without altering original data distribution.

Their study shows that SVM with MKL consistently outperforms single-kernel SVMs
on imbalanced datasets, achieving higher performance metrics. This approach demon-
strates adaptability to various types of imbalanced datasets, including those common in
high-frequency data. Their approach improves classification accuracy and reduces testing
time, an important factor for algorithmic trading. The superiority of the MKL approach over
single-kernel methods is substantiated by statistical evidence [19]. This makes it a good
candidate for algorithmic trading applications dealing with imbalanced data.
50 2.4. Selected topics in time series analysis in markets

2.4 Selected topics in time series analysis in markets

2.4.1 Long memory and persistence

Long memory in financial time series, especially in futures markets, indicates persistent cor-
relations over extended periods, influencing both market dynamics and participant behaviour.
The phenomenon can be visible in microstructure variables like order flow and trade volumes,
which exhibit long-range dependence and multifractality.

The concept, introduced by Hurst [94, 95] and also developed by Mandelbrot [44], de-
scribes processes where dependencies decay more slowly than in short-memory models,
affecting long-term forecasts. Mandelbrot’s fractional Brownian motion (FBM) and fractional
Gaussian noise (FGN) models were seminal in this field [96, 97], offering a modeling approach
that captures the persistent correlations in data [98].

ARFIMA models, introduced by Granger and Joyeux [99], extended this concept by
integrating short-term dependencies, thus providing a more flexible framework for capturing
the details of time series data. The ability of these models to accommodate both long and
short memory has made them essential for accurate modeling in econometrics and finance.

Multifractal detrended fluctuation analysis (MFDFA) is introduced by [29], which is a robust

method for detecting such properties, as it extends traditional DFA by capturing multifractal
features in non-stationary time series.

Gould et al.,[100] probed the long memory of order flow in the FX spot market, focusing
on intraday series without daily data aggregation. They affirm that both the arrival sign and
departure series exhibit long memory (Hurst exponent, H ≈ 0.7). They dismiss the notion
that long memory is an artifact of structural breaks, suggesting its persistence across daily
boundaries on Hotspot FX. Their study underscores the "efficiency paradox" regarding the
unpredictability of return series, suggesting future empirical research on market dynamics
and liquidity correlations.

The long-term memory effects in realized volatility, as demonstrated by Ren et al. [101],
may explain the persistent nature of order book shapes and limit order distributions observed
2. Background and literature review 51

by Bouchaud et al. [9], suggesting that long-range dependence in price fluctuations could be
reflected in the patterns of order flow.

2.5 Advanced statistical techniques

This section covers topics related to Chapter 6.

2.5.1 Kernel two-sample testing

There has been an increasing focus on two-sample nonparametric hypothesis testing, as data
of both high-dimensionality and large size are increasingly being generated in many areas.
As highlighted in 3, we use the well known kernel-based test (KTST), which was proposed by
Gretton et al.[14, 28]. Kernel-based two-sample tests are non-parametric tools for comparing
distributions with increasing applications to time series. The maximum mean discrepancy
(MMD) [14] provides a flexible framework for constructing such tests by embedding probability
distributions into a reproducing kernel Hilbert space (RKHS), where the kernel function
captures the relationship between the data points.
The distributions of the transformed data are compared in the feature space to assess
their similarity. It is particularly useful for analyzing high-dimensional data, where traditional
parametric approaches are not applicable. For time series applications, a key challenge is
accounting for temporal dependence.
Gretton et al. [14] proposed an MMD-based test applicable to weakly stationary processes,
using a block bootstrap to estimate critical values. They demonstrated consistency against
all fixed alternatives under mixing conditions. However, the block size selection remains
challenging for high-frequency data with complex dependency structures.
For ultra-high-frequency data, computational efficiency becomes a major concern. The
quadratic-time MMD estimator used in [14, 102] scales poorly to large sample sizes. To
mitigate this, Jitkrittum et al.[103], proposed a linear-time test based on evaluating the
witness function of the MMD at optimised locations. Their test adapts to local differences in
distributions, which is advantageous for detecting subtle anomalies in high-frequency trading
52 2.5. Advanced statistical techniques

patterns.

Recent work by Schrab et al. [104] introduced an aggregated MMD test (MMDAgg), that
adapts to unknown smoothness of the difference between distributions. This is of particular
relevance for high-frequency data, where the regularity of price processes can vary across
different timescales. Their test provably achieves minimax rates over Sobolev balls, providing
theoretical guarantees for a wide range of time series models. A key finding from [104], is
that MMDAgg maintains high power even when aggregating over a large number of kernels
(up to 12,000 in their experiments).

This suggests the potential for constructing rich kernel families tailored to specific proper-
ties of TAQ data, such as jumps, microstructure noise, and long-range dependence. KTST
has been applied in bioinformatics [105], time series classification [106], fault diagnosis [107],
but only in limited cases on financial data, for example [108] used the method to test jump
components in jump diffusion models. For our work, we are motivated by the work of Gretton
et al. [14, 109] and models from Jitkrittrum et al.[103, 110], who developed the tests that can
validate models against data while preserving interpretability.

2.5.2 Multifractal detrended fluctuation analysis

MFDFA is an effective way to analyze long-term correlations, fluctuations and uncertainties

of non-stationary time series. It was introduced by [29, 111], analyzes the scaling deviations
and eliminates trends in time series.

Also, it can produce the multi-fractal spectrum that exposes variations in the structure
of the fractal over time intervals marked by large and small oscillations [112, 113]. Jiang et
al.[114], surveyed extensive evidence of multifractality in various markets and periods. They
highlight the continued interest in applying MFDFA to finance enabled by new methods and
data availability. The authors emphasise the need for rigorous testing and future directions,
such as the integration of MFDFA with financial networks.

Relevant applications include [115], who found indications of multifractality and long-
range correlations in data from the limit order book. They suggest exploiting these features
2. Background and literature review 53

for realistic market models. Jiang et al. [114], analyzed high-frequency Chinese stocks
with MFDFA, proposing improvements in volatility prediction from multifractal and nonlinear
patterns. Shu et al.,[116] demonstrated MFDFA’s utility for Bitcoin price prediction.

Liu et al. [117] complement these approaches by examining long-term memory dynamics
in crude oil price spreads, demonstrating how MFDFA can be applied to analyze the impact
of exchange rates on commodity markets. In another application, [118] used multifractal
asymmetric cross-correlation analysis (MF-ADCCA) combined with cross-correlation analysis
(CDCCA), to study volatility and correlations between China’s CSI300 and the US SP500
sector stock index.

The results confirmed characteristics of auto-correlation, cross-correlation, and coupled

cross-correlation and reveal that these quantities are not only asymmetric, but are also
influenced by external events. Furthermore, indicated that the strength and persistence of
these correlations vary depending on the distribution of long-range correlations and fat-tailed
distributions.

In related work [119], explored multifractal analysis for early warning signals of key
stock market changes. The authors investigated multiwaveform shape parameters, such as
maximum spectral position, spectral width, and spectral asymmetries, which indicate critical
transitions.

Similarly, the work of [120] examined the volatility of the oil anker freight market and
its characteristics. It used MF-CCA and MF-DPXA (multifractal detrended partial cross-
correlation analysis) to analyze the impact of financial crises and oil prices on volatility
and cross-correlation fluctuations. Results showed stronger multifractality and nonlinear
properties after the crises.

The work of[121] examined the characteristics of fluctuation. An investigation was carried
out to examine the multifractality and asymmetries of the cross correlation between economic
policy uncertainty and agricultural futures prices in China and the United States using
MFDCCA and its asymmetric version.

The results showed that the sensitivity of China’s soybean futures prices to changes in
54 2.5. Advanced statistical techniques

Chinese economic policy is more pronounced than that of other US agricultural futures prices,
leading to a higher level of multifractality and asymmetry.

In the work of [122], the same techniques were used to investigate the cross-correlation
between closing price and trading volume in the Shanghai rebar futures market. The authors
show that prices, volumes, and their cross-correlations exhibit significant multifractal features
and long-range correlations due to fat-tailed distributions. Further multifractal asymmetric
detrended cross-correlation analysis revealed that the cross-correlations have significant
asymmetric multifractality, with the asymmetry being more pronounced for price changes and
volumes in downtrends than uptrends.

In [123], the authors use MF-DCCA to investigate the dynamic relationship between
Japanese yen exchange rates and market anxiety measured by VIX between 1998 and 2016.
The study found that the causal relationship between the Yen exchange rate and VIX is
generally bidirectional, and the cross correlation between the two-time series is multifractal,
with VIX showing a greater degree of multifractality. In addition, cross-correlation coefficients
are considered to be susceptible to economic uncertainties and risks, such as market stress
and an increase n levels of interest rates.

The work of [124] assessed the efficiency of four major crypto currencies, Bitcoin,
Ethereum, Litecoin, and Ripple, by using an asymmetric MF-DFA method applied to 1
hour data. Their work shows significant asymmetric multifractality in the price of cryptocur-
rencies, with higher trends showing stronger multifractality than lower trends. The COVID-19
epidemic has negatively affected the effectiveness of the four currencies, with Bitcoin and
Ethereum the most affected, and the two largest currencies recovered quickly from their
decline in inefficiency by the end of March 2020. The study also shows that the source of
asymmetric multifractality varies between the cryptocurrencies studied.

Multifractality indicates that cryptocurrency prices do not reflect all available information,
which means that trading opportunities can be exploited and the possibility of earning
abnormal profits.
2. Background and literature review 55

2.6 Statistical properties of financial markets

In any scientific principle, the study and summary of statistical properties is an important
prerequisite when attempting to build a theory describing the underlying phenomena [125].

For example, the development of behavioural and agent based models or market -making
models, requires a better understanding of some of these facts.

Stylised empirical facts

Stylized facts can be defined as persistent macrophenomena, which cannot be fully explained
by using microeconomic ideas [126]. Most stylized facts studies have been presented in
calendar time i.e. the time series of interest are indexed, and we extend the study of such
phenomena to other activity clocks.

Furthermore, we examine if this set of properties is indeed common across some of the
most commonly traded instruments, using recent periods in markets. One of the prevailing
questions, is where the surge in activity by high frequency market participants and automated
market makers have caused changes to these empirical facts via the impact of the trading
activity on market structure. In our work, we follow the definitions for the various empirical
facts as defined in the well cited work [8].

In contrast, to most of the literature that focuses from a purely statistical or descriptive
perspective, (in Chapter 6) we try to understand and take advantage of both the structure
and the coupled relationships of these statistical properties to inform the modeling.

Despite a number of studies of the properties of financial instruments being widespread

and popular in academic literature, the availability of high-frequency data and the application
of machine learning for such a study of empirical facts, is fairly recent [8, 127]. The main
statistical properties broadly fall into three main categories: scaling laws, seasonality statistics,
and correlation/dependency behaviour.

For example, the of behavioural and agent-based models or market making models
requires a better understanding of some of these facts. The purpose of Chapter 6, is
56 2.6. Statistical properties of financial markets

to examine the occurrence of these empirical facts in interest rate future contract-linked
statistics. The choice of stochastic (or information) clock is a topic that has been well covered
in academic literature: [3, 39], and it is well known that the various empirical artifacts are
affected by microstructural factors. There is a set of natural unexplored questions:

1. The natural question is how does the choice of stochastic clock affect the stylized fact
of each underlying asset?

2. If we condition on common concepts, do all asset classes exhibit similar behaviour?

3. Are there additional benefits to such transformations?

These questions are addressed in Chapter 6.

2.6.1 Scaling laws and power-law distributions

Bouchaud et al. [128], developed theoretical models linking scaling exponents to order flow
and liquidity dynamics, which are sensitive to information arrival rates. This work provides a
foundation for understanding the power-law behaviour, often observed in financial time series
and motivates our work on orderflow imbalance powerlaw.
In previous work [9], Bouchaud also examines statistical properties of stock order books,
focusing on the distribution of limit order prices and average order book shape. It presents
empirical results from the Paris Bourse and develops models to explain observed patterns.
In similar work Ren et al. [101], analyzes scaling and memory in realized volatility return
intervals for Chinese stocks, showing improved scaling behaviour and long-term memory
effects. However, it’s limited to Chinese markets and a specific time period, potentially limiting
its generalizability. Similar to this direction, Chapter 6 expands into how price movements
and order flow interact in markets.

2.6.2 Volatility clustering and correlation structures

Matia et al. [129] introduced moving window exponent estimation to track non-stationarities in
financial time series. This method is potentially related to changing information clock speeds
2. Background and literature review 57

and can provide insights into the dynamic nature of volatility clustering in high-frequency
data. Krivoruchenko et al. [130], propose multivariate Student probability density functions to
model returns, capturing stylized facts like volatility clustering and leverage effects. They test
the model on 100+ years of Dow Jones data.
The CorGARCH model of Christodoulakis [131], captures persistence in correlations,
aligning with findings of Lundin et al. [37] on long memory in correlation structures. While
Christodoulakis in [131], focused on correlation modeling, we provide a broader analysis of
market microstructure, including order flow imbalance and volatility. In Chapter 6, we use
MFDFA and MFDCCA, offering a more different view of multifractal properties compared to
Christodoulakis’ CorGARCH model.
Hardiman et al. [52] explicitly modeled information arrival as a Hawkes self-exciting
process. Their work uncovered empirical evidence of two linked clock regimes, contributing to
our understanding of correlation structures in financial markets. Shakeel and Srivastava [132],
investigate stylized facts, including volatility clustering and correlation, using S&P CNX
Nifty futures. GARCH models were used to characterize volatility persistence and analyze
autocorrelation in returns.

2.7 Summary

In this chapter, we provide a review of the literature on key background topics related to the
research of the thesis. We covered select foundations of high-frequency data, including mar-
ket microstructure, information efficiency. The concept of information clocks as alternatives
to calendar-time sampling was discussed.
We also explored machine learning applications in algorithmic trading, focusing on kernel
methods, support vector machines, and hybrid generative-discriminative models. This review
also covered long memory and persistence in financial time series, as well as statistical
properties of financial markets such as scaling laws and volatility clustering.
In terms of statistical techniques, select literature on kernel two-sample testing and
multifractal detrended fluctuation analysis, which are the key components of the methodology
58 2.7. Summary

in 6 is also presented. Throughout this chapter, we established the foundations that motivated
subsequent modeling efforts, including market regime classification and the comparison of
distributions of MFDFA statistics. We contextualized our contributions to the literature on
adaptive, interpretable models for multi-scale market dynamics.
Chapter 3

Methodology

This chapter outlines the methodological foundation for the thesis. It covers the rep-
resentation problem, an overview of kernel methods, single- and multi-kernel learning,
and kernel two-sample testing for time series inference. Hidden Markov models for
sequential data modeling and generative embeddings, along with their algorithms are
discussed. We also explore long memory techniques, focusing on multifractal detrended
fluctuation analysis. The chapter concludes with a brief discussion on power-law fitting.
This background supports subsequent experimental case studies. The Appendices C
and B provide technical depth.

59
60 3.1. Introduction

3.1 Introduction

This chapter establishes the methodological foundations for machine learning topics pre-
sented in this thesis. It provides an overview of the key concepts, techniques, and algorithms
utilized in our experimental case studies. We address the challenge of data representation in
machine learning (Section 3.1) and provide an overview of kernel methods (Section 3.2). The
chapter explores supervised and generative learning paradigms (Section 3.3), introducing
hybrid learning, and overviews hidden Markov models (Section 3.4).
We also discuss generative embeddings algorithms (Section 3.5) and provide an in-
depth examination of multiple kernel learning (Section 3.6), including kernel target alignment
(Section 3.6.2). The chapter presents kernel two-sample testing (Section 3.7) and introduces
long memory (Section 3.8), along with algorithms for multifractal detrended fluctuation
analysis (MFDFA) and its cross-correlation variant (MFDCCA). The topics set the stage for
the experimental case studies. Additional theoretical depth is provided in the accompanying
Appendices C and B.

The representation problem

Data representation is fundamental for the effectiveness of machine learning models and
techniques. As data grows exponentially in size, complexity, and variety, modern algorithms
face increasingly challenging obstacles. These challenges must be addressed within the
constraints of computational and memory resources.
Traditionally, the focus was on designing algorithms to extract models from predefined
representations. The prevailing view regarded learning as primarily selecting an adapted
function from an available set. However, this paradigm has shifted over the past decade. The
search for an ideal representation has become a priority. The representation of tasks raises
several important questions. One of the most fundamental is:

1. What is the best way to create an autonomous representation for a specific task?

2. What metrics or criteria can be used to evaluate and rank different representations?
3. Methodology 61

Bengio et al. [133], outline some of the most common priors that can be incorporated in
learning frameworks. We employ kernel-based machine learning techniques to analyze and
model high-frequency financial time series data, drawing on fundamental principles of data
representation and learning theory.

• Smoothness: The expectation that the model’s target function is smooth, meaning
that if two data points are similar, their outputs should be correspondingly close. This
principle is critical for our first case study on generative-discriminative machine learning
models for regime classification. By ensuring the model’s target function is smooth, we
can more accurately predict market regimes, as similar data points (e.g., short-term
price movements) yield similar classifications, enhancing the model’s robustness.

• Multiplicity of factors: This principle addresses the challenge of representing and

modeling numerous variables influencing a target phenomenon. This concept encom-
passes several critical aspects:

1. Feature complexity and high dimensionality, potentially leading to the curse of

dimensionality.

2. Feature selection and interaction modeling.

3. Algorithmic scalability for handling numerous factors.

4. Model interpretability in high-dimensional spaces.

The proposed methodology in this research accounts for the multiple factors influencing
financial market data. The use of distributed representations effectively captures the
complex, multifactorial nature of the data. This is particularly evident in the hybrid
models that combine generative and discriminative approaches, where the distributed
representations are able to capture the complex, multifactorial nature of high-frequency
financial data.

• Hierarchical data organization: The representation problem underscores the impor-

tance of machine learning models capturing complex data patterns through hierarchical
62 3.1. Introduction

structures. Effective data representations should progress from basic features to more
abstract ones across multiple layers, a crucial approach given the often hierarchical
nature of real-world data. Neural networks, especially deep learning models, excel
at this type of data organization, contributing to their success in tasks like image and
speech recognition. While techniques such as multiple kernel learning and composite
kernels attempt to incorporate feature hierarchies into kernel methods, they do not
achieve the same degree of layered abstraction as deep learning approaches. While
techniques like multiple kernel learning and composite kernels attempt to incorporate
feature hierarchies or combinations into the kernel methods framework, they do not
achieve the same degree of layered abstraction as deep learning’s approach.

• Task-overlapping factors: The existence of shared elements across different tasks,

exemplified in methodologies like multitask learning [134] and transfer learning [135].
This is known as transfer learning. Transfer learning is the concept of taking a repre-
sentation that has been learned for one task and applying it to other related tasks.

• Manifold structures: An underlying hypothesis is that tasks can be delineated using

manifolds with dimensional depth lower than that of the initial representation. Manifold
learning rests on the belief that high-dimensional data (such as images and texts)
often lie on or near a lower-dimensional manifold [136, 137]. If this manifold can be
uncovered, the data can be represented more compactly, which can make learning
tasks easier [138]. The concepts revolve around the idea that the original data space
might not be the best or most efficient representation for a given task. For example, data,
which might be non-linearly separable in its original space, become linearly separable
in a transformed, often high-dimensional space [139, 140]. This transformed space
can be thought of as capturing the manifold structure of the data [141]. The kernel trick
allows computation in this space without explicitly defining the transformation.

• Sparsity: Sparsity refers to the idea that only a small fraction of the features or variables
in a dataset are truly informative or relevant for a given task. The representation problem
3. Methodology 63

deals with finding the best way to represent data for a machine learning task. kernel
methods, especially in the context of SVM, naturally embrace sparsity [139, 140]. The
support vectors, which are the data points that define the decision boundary, are
a sparse subset of the entire data set [142]. The representation theorem in kernel
methods (discussed below) allows for this kind of sparse representation, where only
certain data points (the support vectors) are crucial for the decision function.

3.2 Brief overview of kernel methods

This section provides a brief overview of theoretical foundation for the kernel methods used
in the thesis. These methods are fundamental to the hybrid learning approaches and multiple
kernel learning (MKL) techniques as well as kernel two sample testing, which are developed
subsequently. Appendix C provides a more comprehensive and theoretical representation
of each of these topics for completeness. We also guide the reader towards the following
excellent publications [139, 140, 143].

Section C.6.1 introduces the reproducing kernel Hilbert spaces (RKHS), the mathematical
foundation for kernel-based models. Reproducing kernel Hilbert spaces enable the modeling
of complex, nonlinear relationships in financial time series through high-dimensional feature
space mappings. The reproducing property and its connection to positive definite functions
are crucial for understanding how these methods work [143, 144]. The Representer theorem,
discussed in Appendix C.1.2, is central in our approach to regime classification and market
inefficiency detection. It allows us to transform potentially infinite-dimensional optimization
problems into tractable finite-dimensional ones, which is essential when dealing with the high
dimensionality of financial data.

Our formulation of learning problems within the RKHS framework, detailed in Appendix C.1.3,
directly informs our methodology for supervised learning tasks in both case studies. This
framework enables us to develop models that can effectively capture the nonlinear relation-
ships present in financial markets. The various kernel families explored in Appendix C.3.1
form the building blocks of our MKL approach. In particular, the concept of universal kernels
64 3.3. Supervised and generative learning

underpins our ability to approximate complex financial functions and capture subtle market
dynamics.

The distinctions between stationary and non-stationary kernels (Appendix C.3.2), as well
as isotropic and anisotropic kernels (Appendix C.3.3), are important for our model design.
These allow us to tailor our kernel choices to the specific characteristics of high-frequency
financial data, such as time-varying volatility and directional dependencies. Incorporating
kernel target alignment and multiple kernel learning techniques further enhances the model’s
adaptability [145], making it more accurate, as evidenced in Chapter 5.

Appendix C.4 on kernel learning, with its focus on kernels, directly relates to our imple-
mentation of hybrid generative-discriminative models. The optimization problem underlying
SVMs informs our approach to regime classification and provides the first baseline for our
feature selection process in high-frequency data analysis. By grounding our methodologies
in these fundamental concepts of kernel theory, we ensure that our models are effective
in practice and theoretically sound. This foundation is crucial for developing interpretable
and robust models capable of handling high-frequency financial data’s complexity and high
dimensionality.

3.3 Supervised and generative learning

Generative and discriminative models represent distinct paradigms in machine learning,

distinguished by their divergent objectives, inference techniques, and performance char-
acteristics [146]. The discussion concludes with an overview of a hybrid approach that
incorporates elements from both paradigms, forming the core of the first case study. The
primary distinction between these models lies in their approach to modeling the conditional
probability p(y | x; θ) for classification tasks, where x represents feature data and y denotes
class labels. The two approaches differ in their computation of the likelihood p(x, y | θ)
and the posterior probability p(θ | x, y), reflecting variations in the training process. The
posterior probability p(θ | x, y) is the probability of the parameters θ given the data x and y,
and is central to Bayesian inference, where beliefs about the parameters are updated after
3. Methodology 65

observing the data.

Generative models focus on the joint p(x, y), deriving p(y | x) via Bayes’ rule, while
discriminative models directly target p(y | x) [72, 147]. When handling latent variables,
generative models perform joint posterior inference on θ and latent variables z; [72]. On the
other hand, discriminative models estimate the conditional probability p(y | x; θ) directly;
without involving latent variablesz, (these models focus directly on the decision boundary).

These differences give rise to particular advantages and disadvantages for each approach.
The joint distribution is [72, 79],

LGenerative (θ) = p(θ) p(x, y; θ) (3.1)

| {z } | {z }
prior likelihood
N
Y
= p(θ) p(xi , yi | θ) . (3.2)
| {z }
i=1
prior | {z }
data likelihood

However, as discussed earlier, discriminative learning takes the approach of directly maxi-
mizing p(y | x, θ). Modeling the full joint p(x, y; θ) can be wasteful if p(x) does not provide a
predictive value for y. In addition, designing accurate generative models is challenging for
many real-world problems, requiring difficult simplifying assumptions. Formally, let X be the
input space and Y be the output space. Given a training set of example input-output pairs
{(xi , yi )}N
i=1 , our goal is to learn the parameters θ that make the scoring function assign

higher scores to the observed desired pairs (xi , yi ) compared to incorrect alternative outputs
ybi ,

fθ (xi , yi ) > fθ (xi , ybi ), ∀ybi ̸= yi . (3.3)

A common choice for the scoring function is a linear model with extracted features,

fθ (xi , yi ) = θ · ϕ(xi , yi ), (3.4)

where ϕ(xi , yi ) ∈ RD is the so-called feature mapping, which extracts a D-dimensional

66 3.3. Supervised and generative learning

feature vector that captures properties of the input-output pair for prediction.

The advantage of this discriminative approach is that it avoids generative modeling

decisions about joint distribution p(xi , yi ) and instead directly model the posterior p(yi |xi ).
By directly learning to score correct versus incorrect pairs, the model can focus on capturing
predictive relationships between inputs and outputs. This often enables more efficient and
accurate learning for prediction tasks.

The vector of parameters θ , can be estimated by maximizing an appropriate objective

derived from the scoring model. For example, the conditional likelihood of the training data or
a margin loss-based function. This focuses on learning directly the prediction problem, unlike
generative approaches, which can also result in modeling unused distributions.

Figure 3.1: In this stylised visualization, the left side introduces the discriminative approach, simply
drawing a line in the data space to discern between handwritten 0’s and 1’s. The right
boundary can easily separate the two classes without delving into the specifics of their
data distributions. On the contrary, the right side showcases the generative model, which
tries to represent 0’s and 1’s by closely modeling the real-life distributions of these digits
in the data space, highlighting its engagement with the data’s inherent patterns.

Hybrid learning The term hybrid in probabilistic model learning refers to the combina-
tion of generative and discriminative approaches. This typically involves using a mix of a
discriminative and a generative log-likelihood function to optimize predictive performance.

One approach is to create features for discriminative classifiers, leveraging the represen-
tations from generative models; see Lasserre et al. [72]. The Fisher kernel illustrates this
approach, deriving features from the gradient of the log-likelihood function, and is described
3. Methodology 67

below in Section 3.5.1. Another method is the pair DNN-HMM system in speech recogni-
tion [148], where deep neural networks produce posterior probabilities as input features for
HMMs. Multi-task learning [134, 149, 150], represents an additional strategy, concurrently
optimizing generative and discriminative objectives. This work presents a principled frame-
work that reinterprets discriminative training of generative models as standard training of an
expanded model with additional parameters. This allows interpolation between generative
and discriminative extremes through priors imposing soft constraints between parameter
sets. The framework enables the use of unlabeled data in semi-supervised settings while
preserving discriminative performance. Evaluations on synthetic and object recognition
tasks demonstrate improved results compared to solely generative or discriminative ap-
proaches, enhancing models’ ability to generalize from limited data by capturing complex
distributions [151].

3.4 Overview of hidden Markov models

Next we present an overview of topics in hidden Markov models, mathematical details

and implementation of algorithms can be found in Appendix B. We begin by comparing
HMMs with related models, highlighting their unique characteristics and advantages (see
Appendix B.3.1).

HMMs extend simple Markov chains by introducing hidden states, allowing for more
complex temporal dependencies, while also capturing equential dependencies that mixture
models cannot. This is followed by a discussion of various extensions and variants of HMMs
that have been developed to address specific modeling challenges.

These include hidden semi-Markov models (HSMMs) for explicit duration modeling,
factorial HMMs for representing multiple independent processes, and hierarchical HMMs
for modeling nested patterns. The computational considerations of HMM algorithms are
then examined, including their complexity and scalability issues (see Appendix B.3.1). We
discuss the complexity of standard algorithms and introduce techniques like beam search
and variational inference to improve scalability for large datasets or models with many states.
68 3.5. Feature spaces

We also provide practical advice for applying HMMs, covering aspects such as state selection
and parameter initialization (see Appendix B.3.1). The appendix discusses further recent
developments in HMMs, showcasing approaches that enhance model performance and
flexibility.
We present a set of evaluation metrics commonly used to assess HMM performance, help-
ing to understand how to measure and compare model effectiveness (see Appendix B.3.1).
The diverse applications of HMMs across various domains are also explored, demonstrating
their versatility and importance in different fields .
We also provide considerations for applying HMMs in practice, covering aspects such as
state selection, handling of missing data, and parameter initialization. This section includes
guidance on using model selection criteria and techniques to avoid local optima during
training.
Finally, we discuss the concept of feature spaces, which is crucial for understanding how
HMMs embeddings can represent and process data (see Appendix C.4), a relevant topic
for Chapter 6. This overview aims to provide readers with an understanding of HMMs for
completeness, and further references are provided.

3.5 Feature spaces

This section builds on the previous foundation, and outlines HMM- based feature construction
algorithms essential for extracting representations from hidden Markov models.
We present four key algorithms:Fisher score, which measures the information content
of observations concerning model parameters; state space embedding, which quantifies
the expected duration in each state; transition space embedding, which captures state
transition dynamics; and emission space embedding, which highlights state contributions to
observed emissions. These algorithms, each with distinct computational complexities, provide
complementary perspectives on the latent dynamics of HMMs, enabling more robust and
interpretable analysis of complex times series data structures. The mathematical formulations
and complexity analyses of these algorithms are provided to support their application in
3. Methodology 69

modelling.

3.5.1 Fisher score algorithm

The Fisher score quantifies the information each observed variable carries about a statistical
model’s parameters. We used the algorithm below to calculate the observed Fisher score, a
gradient-based measure of an observation’s informativeness regarding model parameters. It
takes a sequence of observations and a calibrated model as input. The algorithm declares
arrays for quantities like observation likelihood, filtering probabilities, and logarithmic proba-
bilities, populated by the α − β algorithm. After initialization, the main loop iteratively updates
log probabilities and Fisher scores for each observation, summing products of logarithmic
and filtering probabilities over all model states.

Fisher score implementation

Algorithm 1 Fisher score algorithm

Input: Sequence of observations o1:n , Model parameters θ
Output: Fisher score ∇ log p(o1:n ; θ)
1: Initialize necessary storage for probabilities and their logarithms
2: for t = 1 to n do
3: Compute log probabilities and their gradients
4: Accumulate contributions to the Fisher Score
5: end for
6: return Fisher score

Complexity analysis of the Fisher score The computational complexity of the Fisher
Score algorithm primarily depends on the calculations of gradients and probabilities at each
time step, leading to a complexity of O(nK 2 ) for n observations and K states.

3.5.2 State space embedding algorithm

The state-space embedding algorithm measures each state’s impact in a hidden Markov
model for a given observation sequence. It maps observations to model states and uses
70 3.5. Feature spaces

the forward-backward algorithm to capture dynamic state probabilities. The SSE quantifies
expected duration in each state, enhancing model interpretability.

State space embedding implementation

Algorithm 2 State space embedding algorithm

Input: Observation sequence O0:T , HMM parameters θ
Output: State-space embedding vector γSS (O0:T ; θ)
1: Initialize arrays for state probabilities and cumulative contributions
2: for each state ζj do
3: for t = 0 to T do
4: Calculate state probabilities using forward-backward algorithm
5: Update cumulative contributions
6: end for
7: end for
8: Compile the contributions to form the embedding vector
9: return γSS (O0:T ; θ)

Complexity analysis for the state space embedding The SSE algorithm’s complexity
predominantly arises from the forward-backward calculations which is O(T × K 2 ), where T is
the number of observations and K is the number of states. The algorithm further aggregates
these probabilities, contributing additional O(T × K) operations, but still dominated by
O(T × K 2 ).

3.5.3 Transition space embedding algorithm

The transition-space embedding (TSE) algorithm quantifies the significance of state tran-
sitions in a hidden Markov model (HMM) based on a given sequence of observations. It
takes an observation sequence and a parameterized HMM model, then traces the transitions
between states across the sequence. The algorithm begins by initializing arrays to store
the probabilities of transitioning from one state to another at each time step. These arrays
are filled using model-derived transition probabilities, calculated through computational tech-
niques that account for the observed data. The algorithm then proceeds through an iterative
3. Methodology 71

loop that updates the transition probabilities. The resulting TSE is a vector that captures the
frequency and probability of each possible state transition, reflecting the temporal dynamics
and dependencies within the HMM model.

Transition-state embedding implementation

Algorithm 3 Transition-state embedding algorithm

Input: Observation sequence O0:T , HMM parameters θ
Output: Transition-space embedding vector ξTS (O0:T ; θ)
1: Initialize ξTS as a zero vector of size K 2 (K: number of states)
2: for t = 0 to T − 1 do
3: for each state pair (ζi , ζj ) do
4: Calculate ξt (i, j) = P (Zt = ζi , Zt+1 = ζj |O0:T , θ) using the forward-backward
algorithm
5: Update ξTS [k]+ = ξt (i, j), where k = i × K + j (flattening 2D index to 1D)
6: end for
7: end for
8: return ξTS (O0:T ; θ)

Complexity analysis of the transition space embedding The TSE algorithm calculates
transition probabilities between state pairs for each time step, leading to a complexity of
O(T × K 2 ). The complexity scales linearly with the number of observations, multiplied by
the square of the number of states.

3.5.4 Emission space embedding algorithm

The emission space embedding (ESE) algorithm calculates the cumulative impact of emis-
sions for each state in a hidden Markov model across a sequence of observations. Given
a sequence of observations and a calibrated HMM model, the algorithm determines how
each state contributes to the observed emissions by computing and aggregating emission
probabilities at every observation point. The resulting ESE vector highlights the relevance of
each state in generating the observed data.
72 3.6. Multiple kernel learning

Emission space embedding implementation

Algorithm 4 Emission space embedding

Input: Observation sequence O0:T , HMM parameters θ
Output: Emission-space embedding vector εES (O0:T ; θ)
1: Initialize εES as a zero vector of size K (K: number of states)
2: for each state ζj do
3: for t = 0 to T do
4: Calculate εt (j) = P (Ot |Zt = ζj , θ) using the emission probabilities
5: Update εES [j]+ = εt (j)
6: end for
7: end for
8: return εES (O0:T ; θ)

Complexity analysis of the emission space embedding This algorithm computes emis-
sion probabilities for each state at each time step, resulting in a complexity of O(T × K).
This reflects linear dependencies on both the number of observations and states.

3.6 Multiple kernel learning

The multikernel learning paradigm is a technique for learning kernel functions, often compris-
ing linear combinations of elementary or weak kernels. MKL aims to address the challenge of
manually creating suitable kernels for specific scenarios. The choice of kernel function is cru-
cial for determining the reproducing kernel Hilbert space (RKHS) and obtaining high accuracy
predictors, especially when dealing with limited training data. Traditionally, kernel selection
involves evaluating potential functions based on their performance on validation data [152].
MKL techniques can autonomously identify optimal combinations of weak-kernels [91]. Cur-
rent MKL methodologies can be divided into two main categories: fixed or heuristic rule-based
techniques, and optimization-based approaches. The former employs established heuristics
to select and combine component kernels, while the latter frames kernel learning as an opti-
mization problem, using mathematical programming to determine optimal kernel combination
weights.
3. Methodology 73

Figure 3.2: Kernels can be combined in various ways using basic operations, producing new kernels
that capture multiple high-level properties. This allows for the expression of complex
structures through the use of fundamental kernel components.

3.6.1 Traditional multi-kernel learning formulation

Multi- kernel learning (MKL) aims to determine a linear combination of M kernel functions,

M
kθ (x, x′ ) = θm km (x, x′ ),
X
(3.5)
m=1

where θ ∈ RM contains the kernel weights, and km : X × X → R. Typically, an ℓp -norm

regularization is imposed on θ ,

M
! p1
p
X
∥θ∥p = |θm | = 1. (3.6)
m=1

This restricts the kernel weights and prevents overfitting.

74 3.6. Multiple kernel learning

Figure 3.3: This illustration demonstrates MKL with the use of a predefined set of kernels where I
am learning an optimal linear (but non-linear is possible too) combination of kernels as
part of the algorithm. For comparison, we show how kernel learning would perform when
using just each of the individual kernels. From the left to right, we have linear, polynomial
and RBF.

Optimization problem in MKL

Given the positive semidefinite base kernels {ki }i∈I , the challenge is to derive a predictor
using a new combined kernel,
X
kθ = θi ki , (3.7)
i∈I
3. Methodology 75

while keeping kθ positive semidefinite. The kernel based optimization problem is formulated
as

X
min Lossn (fu,θ ) + λ |0ui |022 (3.8)
u∈U,θ
i∈I

subject to θ ≥ 0, (3.9)

|0θ|0p ≤ C, (3.10)

with the definitions

u = (ui )i∈I ∈ U = ×i∈I Ui (3.11)

θ ∈ RI (3.12)
n
1 X
Lossn (fu,θ ) = ℓt (fu,θ ) . (3.13)
n t=1

Key algorithms and their characteristics

• Semi-definite programming (SDP): Offers a way to find global optima for convex
optimization problems in MKL. While SDP ensures accuracy, it may not scale well for
large datasets.

• Gradient descent: A widely used method that iteratively adjusts parameters to mini-
mize the objective function. Its simplicity and versatility make it applicable to a broad
range of MKL problems.

• Conjugate gradient descent: Improves upon gradient descent by reducing the number
of iterations needed for convergence, especially beneficial for large-scale optimization
problems.

• Stochastic gradient descent (SGD): An adaptation of gradient descent that uses a

subset of the data for each iteration, significantly speeding up the computation for very
large datasets.
76 3.6. Multiple kernel learning

• Computational trade-offs: The choice among these algorithms involves balancing

the need for computational efficiency with the requirement for accurate and reliable
convergence to an optimal solution.

Understanding model decisions

The interpretability of MKL models, especially in comparison to more opaque models like
deep neural networks, provides significant insights into the data and the decision-making
process of the algorithm.

• Kernel weights: The weights assigned to different kernels reflect their importance and
contribution to the model, offering a direct way to assess the relevance of different data
features or modalities.

• Sparse solutions: MKL techniques that promote sparsity in kernel weights can high-
light the most significant features, simplifying model interpretation.

• Kernel function analysis: Examining the form and parameters of the kernel functions
used can provide insight into the nature of the underlying structure and relationships of
the data.

• Decomposition approaches: Techniques that decompose the MKL solution into

components related to individual kernels can trace the model’s predictions back to
specific data dimensions or features.

• Model transparency: The ability to interpret MKL models enhances transparency and
trust in the model’s decisions, making MKL particularly valuable in applications where
explainability is critical.

One-stage multi- kernel learning

As highlighted in the previous sections, MKL provides a framework for learning an optimal
kernel from a predefined set of base kernels. Let I be a finite index set indexing the base
3. Methodology 77

kernels. Let ki : X × X → R, i ∈ I , denote the ith base kernel. The goal is to learn a
non-negative linear combination of these kernels,

X
kθ = θi ki , (3.14)
i∈I

where kθ is the resulting combined kernel. Since the base kernels are combined with
non-negative weights, kθ remains positive semidefinite. Now, let us consider the function

Xp
fu,θ : X → R given by fu,θ (x) = θi ⟨ui , ϕi (x).⟩, (3.15)
i∈I

where ui resides in the Hilbert space Ui associated with kernel ki , and ϕi : X → Ui is the
corresponding feature map. The kernel-based optimization problem is

λX
min Lossn (fu,θ ) + |0ui |022
u,θ 2 i∈I
(3.16)
s.t. θ ≥ 0,

|0θ|0p ≤ C,

where

u = (ui )i∈I ∈ U = ×i∈I Ui ,

θ ∈ RI ,
n
1X
Lossn (fu,θ ) = ℓt (fu,θ (xt )),
n t=1

√
and ℓt : R → R, t ∈ {1, . . . , n}, are convex loss functions. By letting wi = θi ui , the above
problem is converted into a convex optimization problem. One-stage MKL can also be viewed
as a group p-norm penalized minimization problem,

min Lossn (fw ) + Pen(fw ), (3.17)

w∈W
78 3.6. Multiple kernel learning

where !2/p
X p
Pen(fw ) = ρi ||wi ||p2 . (3.18)
i∈I

This establishes the connection between the regularization of the p norm of the group and
the MKL framework. The group p-norm penalty encourages sparsity at the group level, which
is useful when the number of kernels is large.

Two-stage multi-kernel learning

In two-stage MKL, the kernel learning phase is separate from the predictor learning phase.
This provides flexibility in using different objective functions in each stage. In the first stage,
b∗ = yyT , where
the kernel function is learned by maximizing alignment with the ideal kernel k
y = (y1 , . . . , yn )T collects the training labels. Let f (k) = Abc (k, kb∗ ), where Abc denotes
empirical centered alignment. When restricted to non-negative linear combinations of base
kernels ki , i ∈ I , the alignment maximization problem is
* +
θi ki , kb∗
X
max . (3.19)
θ≥0
i∈I F

This problem is equivelant to the minimization of the Euclidean distance between the com-
b∗ ,
bined kernel and k
2
β X
min θi ki − kb∗ . (3.20)
θ≥0 n2 i∈I
F

In the second stage, the learned kernel is used with a standard kernel method such as SVM
to derive the predictor.

3.6.2 Kernel target alignment

Kernel alignment provides a way to measure the similarity between two kernels, which can
be useful in the design of kernel combinations in MKL. In this section, we discuss the topic of
alignment and target alignment as it relates to kernels. The alignment between two kernel
matrices k, k ′ ∈ RN ×N is defined as
3. Methodology 79

Definition 1. (Kernel target alignment) [153, 154]

⟨k, k ′ ⟩F
A(k, k ′ ) = p , (3.21)
⟨k, k⟩F ⟨k ′ , k ′ ⟩F

where ⟨·, ·⟩F is the Frobenius inner product and || · ||F is the Frobenius norm.

The alignment takes values between 0 and 1, where 1 indicates perfect alignment. In
their seminal work, Cristianini et al. [154] demonstrated that kernel target alignment can be
employed to adjust directions in a kernel matrix to achieve the highest possible alignment
with a linear kernel created with the target vector, which is the target kernel. For example,
in binary classification, the target kernel is defined as YYT where y ∈ {−1, +1}N contains
the class labels. Using the idea of aligning the kernel constructed from data and the target
kernel, Cortes et al. [153] propose an algorithm that learns to combine multiple kernels with
different hyperparameters. Taking inspiration from that idea, we can express the optimization
problem for MKL with kernel target alignment in the following way,

X
min Lossn (fu,θ ) + λ |0ui |022 (3.22)
u∈U,θ
i∈I

subject to θ ≥ 0, (3.23)

|0θ|0p ≤ C. (3.24)

where ki are the base kernel matrices and µ are the combination coefficients. Intuitively, this
finds the convex combination of base kernels that is maximally aligned with the ideal target
kernel for the binary classification task. It provides a principled way to learn an optimal kernel
from multiple candidates. Extensions like centered alignment have also been proposed.

Computational complexity

A key consideration in MKL is computational complexity, which depends on both the number
of training examples n and the number of kernels |I|. For a given optimization algorithm,
the per-iteration complexity and overall complexity will vary. For example, for the QCQP
80 3.7. Overview of kernel two-sample testing

formulation in [155], the overall complexity is O(|I|n3 ) using interior point methods. This
renders MKL intractable for large n and |I|. More recent MKL algorithms based on iterative
block-coordinate descent achieve a per iteration complexity that is linear in |I|. However, the
overall complexity remains linear in |I|, since all kernel weights are updated per iteration.
Stochastic and greedy algorithms can attain sublinear dependence on |I|, by updating only a
single kernel weight per iteration. This enables scalability to very large kernel sets. In general,
devising MKL algorithms that can efficiently handle many kernels or large samples remains an
active area of research. The optimal computational complexity depends on the optimization
algorithm, data size, number of kernels, and other factors. Reducing dependence on n and
|I| is crucial for tackling modern large-scale problems.

Table 3.1: Comparison of multi- kernel learning methods.

Method Type of Combination Optimization Approach Computational Efficiency Scalability

SDP-based MKL Linear Semi-Definite Programming High Moderate
Non-linear MKL Non-linear Iterative Optimization Moderate Low
EasyMKL Linear Efficient Large-scale Optimization Very high Very high
Unified MKL (Kloft et al.) Mixed Norms Smooth Optimization Moderate to High High

3.7 Overview of kernel two-sample testing

This section provides an overview of the statistical framework for the two-sample hypothesis
testing problem. This topic is central to the second experimental study presented in Chapter 6,
where we investigate market inefficiencies in high-frequency financial data using MFDFA.
Our objective is to compare distributions of MFDFA-generated data collections, determining
whether two samples originate from the same distribution. A foundational presentation of this
topic is included in Section C.6 of the Appendix.

3.7.1 Two-sample testing

We aim to determine if two samples from time series data, denoted as P and Q, originate
from the same distribution. The null hypothesis (H0 : P = Q) is tested against the alternative
(HA : P ̸= Q), where the distributions P and Q, characterize different market regimes or
3. Methodology 81

periods in the high-frequency financial data under investigation. The hypothesis testing
framework for Case Study 2 is formulated as

H0 : P = Q, (3.25)

HA : P ̸= Q, (3.26)

where P and Q are the distributions inferred from the two MFDFA data samples; Section 3.7.
In the context of this thesis, these distributions will correspond to different market regimes
or periods of potential inefficiency. To overcome limitations of simple summary statistics in
capturing complex financial time series dynamics, we employ kernel methods that utilize
entire data samples. The kernel function k(x, x′ ), crucial for our analysis in Case Study
2, maps input vectors to a RKHS, allowing for the capture of nonlinear relationships in
high-frequency data. The theoretical foundations of RKHS are detailed in Appendix C.6.1.
For comparing probability distributions within an RKHS, we introduce the maximum mean
discrepancy (MMD); see Appendix C.6.1. In Chapter 6, for a universal RKHS H, we define
the empirical MMD as:

MMD(P, Q) = sup EX ∼ P[f (X)] − EY ∼ Q[f (Y )],

f ∈H:∥f ∥H ≤1

where P and Q are the probability measures corresponding to distributions of our data. The
concept of universal kernels, crucial for this formulation, is elaborated in Appendix C.3.1. In
Chapter 6, we specifically employ the quadratic MMD,

MMDq 2 (p, q) = Ex, x′ ∼ p k(x, x′ ) + Ey, y ′ ∼ q k(y, y ′ ) − 2Ex∼p,y∼q [k(x, y)] .

   

MMD provides richer information by considering interactions between pairs of samples,

crucial for detecting subtle market inefficiencies. Sections C.6.1 describes the linear versus
quadratic implementation. For our experiments in Chapter 6, we use the quadratic MMD with
82 3.8. Long memory and multifractal long memory

an unbiased estimator,

m X m n X n
2 1 X 1 X
MMDu [F, X, Y ] = k(xi , xj ) + k(yi , yj )
m(m − 1) i=1 j̸=i n(n − 1) i=1 j̸=i
m X n (3.27)
2 X
− k(xi , yj ).
mn i=1 j̸=i

This approach allows us to detect subtle differences between MFDFA-generated data distri-
butions, crucial for identifying market inefficiencies and regime changes in high-frequency
financial data. The choice of kernel function, critical for our analysis, is informed by the
discussion in Appendix C.3.1. The practical implementation of this framework , as applied in
our case study, is further elaborated in Chapter 6.

3.8 Long memory and multifractal long memory

Long-term memory and its different forms are important fields of statistical modelling research,
especially in algorithmic trading. Understanding the basic statistical properties of time series
data helps to better model and predict future behaviour. This section examines the differences
between these concepts and discusses other types of long memory. Long-term memory is a
phenomenon in which the current value of a series of times is associated with the value of
the distant past. This correlation gradually decreases over a long period of time, indicating
a persistent trend or pattern in the data. Long memory processes are usually modeled
by fractional integration or fractional difference. Such a modeling approach should ensure
data stationarity. This is crucial for mapping new observations to known examples while
still preserving the predictive power of (time-series) memory, as discussed in detail in [156].
Formally, long memory can be characterized by the autocovariance function γ(k), which
decays as a power law with the lag k ,

γ(k) ∼ k −h as k → ∞, (3.28)

where h is the Hurst exponent, with 0.5 < h < 1 indicating long memory.
3. Methodology 83

3.8.1 Slow hyperbolic decay in the context of long memory

Next, we present a simple definition based on sample autocorrelations:

Definition 2 (Long memory or strong memory). A time series X : [0, ∞) × Ω → R is said to

exhibit long memory, strong memory, long-range dependence, or long-range correlations if
its sample autocorrelation function ρ̂(k), defined as per [157],

n−|k| 
1 X  
ρ̂(k) = X(ti , ω) − X̄ X(ti+|k| , ω) − X̄ , (3.29)
n − |k| i=1

1 Pn
where X̄ = n i=1 X(ω, ti ) is the sample mean of the process X, decays at a hyperbolic
rate as k tends to infinity. More specifically, if there exists some 0 < δ < 1 such that
ρ̂(k) ≈ k −δ , then the decay of ρ̂(k) is said to be slow or hyperbolic, implying non-summability
of the autocorrelation function and indicating the presence of long memory.

A time series X(t) is said to exhibit long memory if its autocovariance function Γ(k)
decays in a hyperbolic manner as the lag k increases,

Γ(k) ∼ c k −δ , as k → ∞, (3.30)

where 0 < δ < 1 is the long memory parameter and c is a constant. The hyperbolic decay
of the autocovariance function with an exponent 0 < δ < 1 indicates that the correlations
between observations decay slowly and persist for a long time. This is the defining character-
istic of long memory or long-range dependence in time series data. The equation defines the
autocovariance function γ(h) for a long memory process as decaying as a power law. The
exponent determines how quickly the autocovariance decays; the smaller the exponent, the
slower the decay and stronger the long memory.

Multifractal long memory

Multifractal long memory is characterized by long-memory correlations exhibiting a multiscale

structure, where the correlation structure varies across different time scales. Unlike simple
84 3.8. Long memory and multifractal long memory

long memory with uniform scaling, the data show different scaling behaviours at various time
horizons. Long memory refers to long-term correlations following a single scaling behaviour,
while multifractal long memory exhibits multiple scaling behaviours across different time
scales. The data display fractal or self-similar properties to varying magnifications of the time
axis. Mathematically, multifractal long memory is characterized by the multifractal spectrum
f (α), describing the distribution of scaling exponents α across scales. In a multifractal
process, f (α) is a continuous function, indicating multiple scaling behaviours in the data.
Different parts or scales of the series exhibit different degrees of long memory. This is
characterized by a scale-dependent Hurst exponent, Hh (q, s), where q is the order of the
statistical moment and s is the scale of the time series. The q-th order structure function, or
fluctuation function, F (q, s), measures the qth moment of the increments of the time series
over a scale s. For a stationary time series X(t), the q-th order structure function can be
defined as,

Definition 3 (Multifractal long memory).

Fq (s) = E[X(t + s) − X(t)|q ], (3.31)

where E denotes the expectation, t is time, and s is the scale. Hh (q, s) captures the scaling
behaviour of this structure function. In a multifractal series, the structure function scales as a
power law over the scale s, which can be different for different moments q . This power-law
behaviour is characterized by the generalized Hurst exponent,

Fq (s) ∼ shq (s) , as s → ∞. (3.32)

This equation indicates that the structure function Fq (s) scales as a power of the scale
s, with the power given by hq (s). When hq (s) varies with q and/or s, it indicates multifractal
scaling. The autocovariance function γ(q, k, s) decays as a power law with lag k , and in a
3. Methodology 85

multifractal long-memory process, this decay is different for different moments q and scales s,

γ(q, k, s) ∼ k −hq (s) , as k → ∞. (3.33)

In the above equation, the Fq (s) term captures the scaling behaviour (structure function) at
different scales s, and the hq (s) term represents the multifractality and the varying scaling
across these scales. The autocovariance function γ(q, k, s) reflects these same characteris-
tics, but in the time domain, showing how correlations decay with the lag k .

Characteristics of long memory

In this section, we will give a very brief and informal overview of some of the characteristics of
long memory. We will refer the interested reader to the following references for more formal
definitions [157, 158]. We focus on two areas specifically and attempt to provide informal
definitions for completeness.

Anomalous trends and cycles and self-similarity Financial times series often exhibit
anomalous local trends and periodicities that dissipate and reappear at varying frequencies
and lengths. This property implies that scaling time independently modifies the probability
distribution [157]. The observed anomalies likely result from random fluctuations, external
shocks, and non-linear dynamics, causing abrupt changes and long-term patterns [159]. The
self-similarity of the time series suggests that it is governed by scale-invariant underlying
laws. This makes it difficult to predict the behaviour of time series in the future, but it can be
used to analyze its past behaviour and identify possible patterns [157]. The identification
of long-memory storage is complicated by the fact that typical memory characteristics can
be mistaken for nonstationary components. However, it is essential to identify and assess
potential long memory components to accurately conclude nonstationarity.
86 3.8. Long memory and multifractal long memory

Other types of long memory

Fractionally integrated processes are a class of long-memory processes where the degree
of integration, δ , is a non-integer value, typically falling within the range 0 < δ < 1. Such
processes demonstrate long-range dependence and are frequently employed in modeling
complex data such as financial market movements. The mathematical characterization of a
fractionally integrated process can be encapsulated by the fractional differencing operator Dδ .
Specifically, a time series X(t) is said to be fractionally integrated of order δ if the following
holds,
X(t) = Dδ Z(t), (3.34)

where Z(t) is a stationary process. Long-range dependence with heavy tails refers to a time
series’ distribution tails decaying slowly. This property is useful for modeling phenomena like
Internet traffic and stock returns. A time series with heavy tails is characterized by its tail
index β , measuring the tail distribution’s decay rate. Heavy tails and long-range dependence
are present when β > 2.

3.8.2 Statistical techniques for modeling long memory

Several statistical techniques have been developed to model long memory and multifractal
long memory. For example, the ARFIMA model is a popular choice for modeling long memory,
while the MF-DFA method is commonly used for analyzing multifractal long memory. Other
methods similar to MF-DFA include wavelet leaders and detrended fluctuation analysis (DFA).
We use MFDFA and MFDCCA in Chapter 6.
3. Methodology 87

Figure 3.4: This figure compares four stochastic processes: Gegenbauer ARMA, ARFIMA(, 0, 2),
fractional Brownian motion, and GARCH(1,1). Time series plots show complex patterns
in long-memory processes (left two columns) versus short-memory processes (right
two columns). ACF plots demonstrate slow decay for long-memory and rapid decay
for short-memory processes. Gram matrix heatmaps reveal diffuse autocorrelation in
long-memory processes and localized correlation in short-memory processes. These
distinctions are crucial for time series analysis and forecasting.

3.9 Overview of MFDFA and MF-DCCA

We present an overview of the computations for each of the MFDFA variables for all of the
symbols. The detailed algorithm is presented below in Section 3.9.1.

1. MFDFA begins by obtaining the fluctuation function Fq (s), where q is the moment order
and s is the scale of the analysis.

2. The generalized Hurst exponent h(q) is then determined as the slope of the log-log
plot of Fq (s) versus s,
d log Fq (s)
h(q) = , (3.35)
d log s

3. The singularity strengths, or Hölder exponents, α are given by the derivative of h(q)
88 3.9. Overview of MFDFA and MF-DCCA

with respect to q ,
dh(q)
α= (3.36)
dq

4. The multifractal spectrum f (α) is subsequently calculated via the Legendre transfor-
mation of the function h(q),
f (α) = qα − h(q). (3.37)

The relationship in Equation 3.37, establishes the correspondence between the MFDFA vari-
ables and the multifractal spectrum. However, it should be noted that the reverse process i.e.
that derives all MFDFA variables from the multifractal spectrum is not feasible due to the loss
of information in the process of calculating the spectrum.

3.9.1 Algorithms for MFDFA and MFDCCA

Our study employs multifractal detrended fluctuation analysis (MFDFA) and multifractal
detrended cross-correlation analysis (MFDCCA) to analyze the multifractal properties and
cross-correlations in financial time series.

Multifractal detrended fluctuation analysis

The implementation of the MFDFA algorithm, adapted from [111].

3. Methodology 89

Algorithm 5 MFDFA algorithm

Input: Time series x(t) of length N
1: Profile Calculation: Compute the profile Y (i)

i
X
Y (i) = (xk − ⟨x⟩), i = 1, 2, . . . , N (3.38)
k=1

Benefit: Removes noise and highlightsjunderlying

k trends.
2: Segmentation: Divide Y (i) into Ns = Ns non-overlapping segments. Benefit: Enables
localized analysis.
3: Local trend removal: Fit a polynomial of order m and calculate the detrended series.
Benefit: Focuses on inherent fluctuations.
4: Fluctuation function: Compute the variance for each segment and average to get Fq (s):

Ns 
" #1/q
1 X q/2
Fq (s) = Fi2 (s) (3.39)
Ns v=1

Benefit: Quantifies data variability.

5: Scaling exponent: Determine the scaling behaviour by fitting log Fq (s) vs. log s:

log Fq (s) = h(q) log s + constant (3.40)

Benefit: Identifies long-term dependencies.

90 3.10. Summary

Multifractal detrended cross-correlation analysis

Algorithm 6 MFDCCA algorithm

Input: Two time series x(i) and y(i) of length N
1: Profile Calculation: Compute profiles X(i) and Y (i):

i
X i
X
X(i) = (xk − ⟨x⟩), Y (i) = (yk − ⟨y⟩) (3.41)
k=1 k=1

Benefit: Detects underlying trends and co-movements.

j k
N
2: Segmentation: Divide X(i) and Y (i) into Ns = s segments. Benefit: Enables
localized analysis.
3: Local trend removal: Fit polynomials and detrend the series. Benefit: Enhances
predictive accuracy.
4: Cross-correlation fluctuation function: Compute Fxy2 (s, v):

s
2 1X
Fxy (s, v) = [X((v − 1)s + i) − Xv (i)][Y ((v − 1)s + i) − Yv (i)] (3.42)
s i=1

Benefit: Quantifies co-movement.

5: Average fluctuation function: Average over all segments to get Fq (s):

#1/q
1 2N
"
Xs
Fq (s) = F 2 (s, v)q/2 (3.43)
2Ns v=1 xy

Benefit: Provides a comprehensive measure of cross-correlation.

6: Scaling exponent: Determine Hxy (q) by analyzing Fq (s) vs. s:

Fq (s) ∼ sHxy (q) (3.44)

Benefit: Reveals long-range cross-correlations.

3.10 Summary

This chapter establishes the methodological foundations for analysing high-frequency financial
time series data, addressing the challenges of modeling long-range dependencies and
complex nonlinearities [160]. It also covers the representation problem, kernel methods,
learning paradigms, and hidden Markov models, introducing algorithms for constructing
3. Methodology 91

generative feature spaces. It also explores multiple kernel learning approaches and kernel
target alignment, with a focus on kernel two-sample testing to investigate market inefficiencies.
Finally, it introduces the maximum mean discrepancy for measuring distribution distances
in an RKHS. Furthermore, the chapter examines long memory and multifractal concepts,
including MFDFA and MFDCCA techniques for analyzing multifractal properties and cross-
correlations in financial time series. This theoretical framework supports the development
of interpretable and effective models for high-frequency financial data in subsequent case
studies.
Chapter 4

Data and their statistical transforma-

tions

This chapter discusses the data used in the thesis and the building blocks for data-
driven methodologies and transformation techniques used in this thesis. The chapter
also introduces methods for the automated creation of class labels from experimental
market data and feature analysis techniques such as correlation analysis, RadViz, and
t-SNE. These methods aim to capture intrinsic imbalances and evolving market trends in
high-frequency datasets, supporting the development of machine learning algorithms for
financial applications.

93
94 4.1. Introduction

4.1 Introduction

This chapter establishes the foundation for this thesis’s data-driven methodologies and
statistical techniques. We begin by describing the challenges and opportunities of high-
frequency financial data, particularly trade and quote (TAQ) data derived from limit order
books [161, 162]. The digitization of limit order books has enabled the precise recording
of time series data for orders placed, modified, canceled, or executed, necessitating inno-
vative statistical and machine learning methods to model these complex, high-frequency
processes [163].

At the core of this research is the concept of information clocks, which offers a framework
to examine the microstructural dynamics of financial markets [3, 49]. We introduce various
types of information clocks, such as volume, tick, and dollar, and discuss why they are
better suited to capture algorithmic trading activity [53]. A key contribution of this chapter is
the methodology to address the classification challenge associated with highly imbalanced
high-frequency financial data [156]. We develop a set of automated labeling techniques
based on dynamic thresholds derived from empirical quantile estimates, which aim to capture
intrinsic imbalances and evolving market trends [164].

We also outline the data processing pipeline, detailing the transformation of raw financial
information exchange (FIX) messages into feature sets (which are sampled using information
clocks). This pipeline forms the basis for the data used for the empirical studies in Chap-
ters 5& 6. To improve our understanding of the high-dimensional feature space, advanced
visualization techniques such as t-SNE [165] and RadViz [166] are used. These methods al-
low us to analyze the separability of features and guide the experimental design. We present
results from applying these techniques to both raw and kernelized features, demonstrating
how kernelization can improve visual feature discrimination [167, 168]. In the accompanying
Appendix A, we present a simple mathematical framework for the limit order book (LOB),
discussing trade and quote data, and some of the LOB mechanisms.
4. Data and their statistical transformations
95

4.2 Background

This section provides related research on the challenges and methodologies used in handling
the data data. The focus is on noisy label challenge, sequence learning, and advanced
visualization techniques such as t-SNE and RadViz [165, 166, 169–171].

Addressing noisy labels in TAQ data TAQ data can present a significant challenge due to
noisy labels, which can arise from various sources such as market anomalies and erroneous
trades [171]. These noisy labels substantially affect model performance and require robust
classification methods. Frenay et al. [171], emphasized the importance of noise-tolerant
learning algorithms in this specific context to mitigate misclassification risks. Liu et al. [169],
addressed the challenge by implementing importance reweighting methods, which assign
weights to training samples based on their estimated reliability. This technique improves
model robustness, as detailed in their study on temporal Markov dynamics in graph-based
visualizations.

Our strategy, discussed in Chapter 5, draws partial equivalence from these approaches,
integrating them into our hybrid discriminative-generative framework (elaborated in Chapter 3).
This framework combines joint probability distribution modeling to correct noisy labels with
improved classification accuracy, effectively handling the complexities of the data. Huang et
al. in [172], motivates our development of the automated labeling approach.

The authors demonstrate that in types of data (like TAQ data) where perfect class
separation may be challenging due to noise and complex (market) dynamics, a statistical
support vector machine framework can effectively integrate the flexibility of SVMs with
the robustness of Bayesian methods. This suggests that our technique of using dynamic
volatility thresholds for automated labeling to balance the need for adaptability to evolving
market conditions to establish statistically sound classification boundaries. Also, our hybrid
discriminative-generative framework for draws inspiration from the Forward-Decoding Kernel
Machine approach in [173]. They combine large-margin classifiers with hidden Markov models
to capture the discriminative power of kernel methods and the sequential dependencies in the
96 4.2. Background

data. Similarly, our automated labeling technique using dynamic volatility thresholds adapts
to market conditions, similar to how FDKM conditions state transitions on observed data.

Incorporating sequence learning Dredze et al. [170], discuss how sequence learning
algorithms can be adapted for multiple labels can be particularly effective for high-frequency
financial data, where trading signals often have multiple plausible interpretations due to
market noise and volatility. Our approach adapts this concept by using dynamic volatility
thresholds to implicitly determine label probabilities. As volatility changes, these thresholds
adjust automatically, refining the labeling process over time. By dynamically adjusting to the
market conditions, our approach offers improved accuracy over static, single-label methods
in high-frequency financial environments.

By considering volatility (and therefore label probability) we can refine the process over
time. Hence, these methods can better handle the noise in the data. Also, as discussed in
Chapter 3 and in Chakrabartty et al. [173], integrating generative features with discriminative
models allows for better handling of noisy labels. Our approach improved precision over
single-label fixed methods in this context [170].

Visualisations with t-SNE and RadViz

Learning discriminative features for regime classification involves handling noisy, high-
dimensional data with variable automated labelling. Multiple kernel learning and other
methods can learn robust features from raw financial data, improving signal classification
despite noisy labels. Visualizing these learned features provides insights into model be-
haviour and the impact of noise, which also helps in refining the labeling and modeling
processes. Since the feature set for the case study in Chapter 5 is multidimensional, we use
t-SNE [165, 174] and the RadViz mapping method to project the N-dimensional space set
into a 2-dimensional plane in which the effect of each dimension is proportional to the sum of
all dimensions’ effects.
4. Data and their statistical transformations
97

RadViz RadViz is a non-linear multidimensional visualization technique that can present

data described by multiple dimensions (typically at least three) in a two-dimensional plane. It
was introduced by [166, 175]. The variables are depicted as anchor points evenly spaced
around the circumference of a unit circle. Data instances are represented as points within
a circle, and their positions are determined by a physics-inspired metaphor: each point
is held in place by springs connected to variable anchors. The stiffness of each spring is
proportional to the value of the variable to which it is linked, and the spring forces are equal
at the equilibrium point. In Chapter 5, RadViz helped reveal the clustering and grouping of
features, while t-SNE provided insight into the overlap and similarity of the features.

t-SNE t-SNE is a technique for embedding high-dimensional data in lower dimensions,

(such as 2D or 3D) for visualization [165]. The method models similarities between data points
in both high- and low-dimensional spaces, minimizing KL divergence between distributions
with gradient descent. This nonlinear transformation reveals structure while preserving local
relationships. We apply t-SNE to the raw features and kernelized features, exposing the value
of our kernelization approach. Converting pairwise distances into joint probabilities, preserves
local similarities through the relative positioning of data points. Since the t-SNE objective
function is non-convex, the algorithm’s output may vary depending on the initial conditions.
The methodological insights of [176, 177] are directly applied to enhance the visualization and
interpretation of high-frequency financial data. Both RadViz and t-SNE visualizations support
data mining approaches, improving model interpretability and classification performance.

4.3 Information clock

An information clock is the measure of market activity that takes into account not only the
passage of time but also the amount of information that arrives in the market. It can be
considered a non-conventional timing mechanism. We assume that it denotes the information
flow (in some units) at (calendar) time t. Then we can define information time as the integral of
information flow from the start of our observation period (which is denoted by t0 ) to some time
98 4.3. Information clock

tf . Let us now define the concept of an information clock formally, based on the cumulative
traded volume, (i.e. volume clock ) or the number of trades, (i.e. tick clock ), or the market
value of the cumulative volume traded (i.e. dollar clock ).

Definition 4 (Information clock). Consider three stochastic processes X : Ω × [t0 , tf ] → R,

V : Ω × [t0 , tf ] → R+ , N : Ω × [t0 , tf ] → N: the stochastic process X(ω, t) is the price of
an instrument at time t under the outcome ω . The stochastic process V(ω, t) is the volume
traded at time t in the outcome ω , and N(ω, t) is the number of trades at time s in the
outcome ω . The cumulative processes

Z ts Z ts
Vc (ω, ts ) = V(ω, t)dt, Nc (ω, ts ) = N(ω, t)dt, (4.1)
t0 t0

are the cumulative volume traded up to time ts and the cumulative number of trades up to
time ts , both under the outcome ω . Given a volume threshold VT̃ and a trade count threshold
NT̃ , we define two stopping times τV and τN ,

τV = inf{s ≥ t0 : Vc (ts , ω) ≥ VT̃ }, τN = inf{ts ≥ t0 : Nc (ts , ω) ≥ NT̃ }. (4.2)

For both definitions, it is assumed that the set is nonempty and otherwise takes the value t.
These are the stopping times with respect to the filtration (Ft )t∈[t0 ,tf ] . For completeness, we
state that τV represents the time when the cumulative volume traded first reaches or exceeds
the threshold VT within the interval [t0 , tf ], while τN represents the time when the cumulative
number of trades first reaches or exceeds the threshold NT within the interval [t0 , tf ]. The
information clocks τV and τN indicate the times at which we sample the stochastic process
X, either when the cumulative volume of trades reaches VT or when the number of trades
reaches NT within the interval [t0 , tf ].

Now, we can define the increments of the information clocks as follows. For any ts ≥ t0 ,
let n = max{k ∈ N : τV (k) ≤ ts } and m = max{k ∈ N : τN (k) ≤ ts }; then

∆τV (ω, s) = τV (ω, n + 1) − τV (ω, n), ∆τN (ω, s) = τN (ω, m + 1) − τN (ω, m). (4.3)
4. Data and their statistical transformations
99

Here, ∆τV (ω, ts ) is the increase in the volume clock time at which the cumulative traded
volume reaches the next volume threshold after time ts with the result ω . ∆τN (ω, ts ) is the
increment in the tick clock time at which the cumulative number of trades reaches the next
trade count threshold after time ts under the outcome ω .

Why does the information clock matter? The starting point is that the arrival of trans-
actions and orders contains information. There is extensive research on the topic but the
works of Easley et al. [35], and Geman [49] are seminal in introducing the topic of information
clocks and its applications. Integrated traders strategically schedule their trades around
volume-clock events to use their information advantage. Black et al. [178], develop a new
measure of trade time clustering to capture how trades cluster over time. This clustering
implies that trades are not evenly distributed but are concentrated together.

Information clock as time dillation Market participants may interpret the same market
data differently, similar to how observers in physics can perceive elapsed time differently due
to differences in relative velocities or gravitational potentials. This is analogous to algorithmic
trading, where trading algorithms process market events at varying speeds based on their
strategy and latency. Although the speed of light is constant in relativity, it can lead to differing
perceptions of events across observers. In markets, despite constant objective data, trading
strategies interpret data influenced by their activity "clock".

1. Accurate sampling and model calibration: Sampling based on volume or a set

number of events helps reduce nonstationarities and potentially help recover normality
in returns [1]. Models calibrated on event time exhibit can be (more) stable and robust,
especially compared to calendar time-based models which can be affected episodes of
irregular market dynamics.

2. Reducing adverse selection: Easley et al. [3], discuss how aligning trading activity
with actual market events reduces the likelihood of transactions with informed traders,
thus reducing adverse selection. Event clocks allow for a better predictability of short
100 4.3. Information clock

term volatility [179].

3. Adaptation to market dynamics: Algorithms using volume- or event-based triggers

synchronize their activity with the underlying market patterns. Trading activity and
order flow are not randomly distributed but concentrated around certain time intervals.
There is a periodic volume clock driving the trading behavior. Volume/activity peaks
at the open and close of trading, and is lower in the middle of the day. This creates a
U-shaped intraday pattern.

4. Reduction in over trading: During periods of reduced liquidity, event-driven mecha-

nisms can help reduce over- trading, in a way safeguarding against adverse transac-
tional costs which are inherent in time-driven strategies. Ultimately, volume or event
clocks can ensure execution trajectories that are more uniform.

5. Microstructure understanding: Looking at relationships under an event clock can

help determine causal relationships between individual trades and price returns.

4.3.1 Related work

The works of Easley et al. [3, 53], are seminal for the application of information clocks. The
dynamics of trade arrivals were modeled on the basis of a microstructure framework. A
sample of 16 NYSE stocks from 15 years of transaction data to estimate trade dynamics
from both informed and uninformed sources. The daily number of buys and sells for each
stock informed this analysis. A point process for each type of investor was developed, with
an intensity parameter conditional on the specific historical information of each investor.

These methodologies are seminal for the application of information clocks, which provide
a more natural timescale for the analysis of high-frequency financial time series data that is
challenged by complex dependencies, non-stationarities, and stochastic timings. Strong daily
autocorrelations were identified in all trade series, with the arrival rate of uninformed trades
being negatively influenced by past rates of informed trades. Furthermore, the dynamics of
informed traders demonstrated a weak response to the conditional intensity of uninformed
4. Data and their statistical transformations
101

traders. These findings contributed to forecasts of market liquidity aspects and probable
informed trade rates. The application of these types of forecasts is discussed, with an
emphasis on a practical setting and academic research.

4.4 Automated class label construction

This section develops an automated model-based approach for constructing class labels re-
quired by discriminative classifiers, which are not directly observed in high-frequency financial
data. The method uses dynamic thresholds based on empirical quantile estimates, inspired
by concepts in [156]. This methodology is applied in Chapter 5. The work is motivated
by [19], who propose that algorithmic-level approaches directly modify learning algorithms
to handle imbalance, avoiding data manipulation. These approaches can optimize the loss
function, focus on minority classes, and adapt to the characteristics of the data set. This is a
computationally efficient approach that is well suited for large data streams. The proposed
approach employs three label-generating mechanisms with different parameterizations, creat-
ing six distinct label sets. These mechanisms consider recent trends and short-term volatility
within intra-day datasets, using quantile-based thresholds to adapt to market dynamics.
Comparisons among label sets provide insight into behavioral patterns and predictive power.
The approach demonstrates robustness and adaptability, with the potential for extension to
various asset classes, market conditions, and trading strategies. Utilising empirical quantile
based thresholds provides advantages, for example:

1. Adaptive empirical quantile threshold approach is robust to noise and anomalies and
automatically adjusts labels based on evolving volatility dynamics;

2. Ability to generate both multiclass and binary labels, so that it has general applicability;
see [180].

3. Scales efficiently on large and diverse high-frequency market datasets where manual
labeling is infeasible [181];

4. The technique captures intrinsic imbalances and supply changes [156, 164].
102 4.4. Automated class label construction

Furthermore, the approach adopted is based on the trade or tick clock, meaning that returns
are calculated using trade times and the corresponding traded prices at trade events, the
benefit of this time dilated approach is discussed in [53]. Additionally, to ensure that the
quantile thresholds are locally adaptive and smoothly transitioning, we used either an expo-
nential or a moving average sliding window to obtain a smoothed observed traded return
trend series before constructing the quantile thresholds applied to produce the target label
sequence labeling. Therefore, for a given day, given the intra-day trade time observed return
series {xti }, the smoothed signal used to construct the label sequence is

sEWMA (ti ) = wxti + (1 − w)sEWMA (ti−1 ), (4.4)

where xti = ξi is the log return at time ti , pti is the price at time ti , sEWMA (ti−1 ) is the
exponential weighted moving average signal at time ti−1 , and the smoothing weight w is a
value in the interval [0, 1]. To detect a statistically significant deviation from this smoothed
return trend, we define a type of generic labelling rule such that the label oti = −1, 0, 1 is


if xti > sEWMA (ti ) + cvti ,




 1,



if xti ∈ [sEWMA (ti ) − cvti ,

 0,

def
Otsi = (4.5)




 sEWMA (ti ) + cvti ],




−1,

if xti < sEWMA (ti ) − cvti .

To determine a time-varying adaptive threshold, one can use the observed volatility of the
underlying instrument returns. For example, one can utilize the range-based realized volatility
estimator vti at time ti , where one can use, for instance, the Parkinson or high-low range
volatility given, for the OHLC data for the stock price at some desired sampling resolution
4. Data and their statistical transformations
103

(say 1 sec) such that with N time periods,

(pO H L C
t−N , pt−N , pt−N , pt−N ),

(pO H L C
t−N −1 , pt−N −1 , pt−N −1 , pt−N −1 ),
..
.

(pO H L C
t , pt , pt , pt ),

the range-based realized volatility estimator is

v !2
N
pH
u
u 1 X
vt = t log iL . (4.6)
4N log 2 i=1 pi

The selection of the multiplier c ∈ R+ dictates the quantile level when selected as a z -score
or t-score.

To illustrate the previously defined generic labeling framework, one can define the type of
labeling strategies that are useful in practice as follows. Furthermore, an example is provided
as an illustration in Figure 4.1.

• Point-in-time: at every point in the trade clock, as long as the current trade price
exceeds a predetermined threshold above or below the previous trade price, an event
is constructed that constitutes a label. Let τeL and τeU be the lower and upper thresholds;
then, 




 1, if Pi > Pi−1 (1 + τeU )


def
OtBi = or Pi < Pi−1 (1 − τeL ) (4.7)





0,

otherwise.

−1,




 if Pi > Pi−1 (1 + τeU )


def
Otmulti
i
= 1, if Pi < Pi−1 (1 − τeL ) . (4.8)





 0,

otherwise.
104 4.4. Automated class label construction

Figure 4.1: Illustrative time series with the visualization of some stylized boundaries for the labeling
mechanism framework. At every point in the trade clock, we observe what the moving
average trade price was, and as long as the current trade price exceeds the empirical
quantile threshold, computed using historical volatility, an event that constitutes a label
is constructed. The binary label indicator function can be seen with the green and red
labels.

• Moving-average based: construct a moving average of the trade prices for a window
k . At every point on the trade clock, as long as the current trade price exceeds a
predetermined threshold above or below the moving average trade price, an event
that constitutes a label is constructed. The up-and-down threshold is constructed as
an empirical quantile using estimated historical volatility. The historical volatility is
estimated as the average intraday historical volatility on a parametric window of the
number of days. Let (νL ) and (νU ) be the lower and upper thresholds, respectively;






 1, if Pi > Pi−1 (1 + νU )


def
Otmav
i
= or Pi < Pi−1 (1 − νL ) (4.9)





0, otherwise.


By incorporating multiple parameterizations, the proposed labeling methodology offers a

4. Data and their statistical transformations
105

flexible and comprehensive framework to capture the complex dynamics of high-frequency

financial markets. This approach enhances the robustness and generalizability of the HMM-
SVM-MKL framework, enabling more accurate and nuanced predictions of market regimes
and trading opportunities.

The settings for the automatic labelling algorithm (for the experiments in Chapter 5) are
given in 4.6.1.

4.5 Data and methods

This section provides details of the irregularly spaced data used in both Chapters 5 & 6.
We analyze TAQ data, initially collecting raw data and then defining and selecting a set of
information events. To ensure universality in our findings, a large number of datasets and
instruments are required.

Generic processing pipeline for data

In algorithmic trading research, the data processing pipeline is a critical infrastructure com-
ponent that handles input, output, and business logic for data transformation. This section
describes a generic transformation and training process used to convert Financial Informa-
tion Exchange data into feature sets sampled using information clock techniques, which is
then extended in each of the Chapters 5 & 6. The following section outlines the workflow,
explaining the relationships between each procedural step to ensure functionality across the
entire pipeline.

We sample the market at both regular intervals (to study the effects of changing the
information clock) and irregular intervals (as trades have such arrival rates, and to study
effects under the information clock).

We follow a 3-stage approach:

1. We process FIX messages (the financial information exchange (FIX) protocol is a

series of messaging specifications used to price and trade data), and construct TAQ
106 4.5. Data and methods

data (trade and quotes). Steps A and B in the figure below.

2. For each of the experiments, we formulate various statistics that describe the market
microstructure (e.g., price dynamics, interarrival time, volatility, volume, liquidity, order
flow). This is step C.

3. These are then sampled at different information clocks, establishing robust data captur-
ing microstructure and information dynamics. This is step D.

4. In the final stage, we derive times series of variables from the time series of statistics
derived in stage 1. This is step E in Figure 4.2.

5. This data is then used for each of the experiments in Chapters 5 & 6.

The raw data set comprises top-of-the-order-book and trades data recorded during 2017
for the FTSE stocks and 2018 for the interest rate futures contracts. Exchanges typically
publish order book snapshots at intervals of 5 to 200 ms, depending on market activity.
Creating uniformity and comparability within a large data set poses a challenge. We define
the trade price for an information clock period as the information-event-weighted average
price computed over all trades within that window. If no trade occurs during a window, the
previous valid price observation is used. Trade sizes have different denominations across
markets.
4. Data and their statistical transformations
107

fx

| {z } | {z } | {z } | {z } | {z }
A. B. C. D. E.
FIX TAQ Constructing Information Data for
messages Data microstructure clock experi-
variables sampling ments.

Figure 4.2: We initiate our process with the assembly of a TAQ dataset (B) from exchange message
files (A), featuring FTSE Stocks in 5 and futures contracts in 6. The following step involves
the generation of microstructure variables (C). Subsequently, we sample the variables
using different information clocks (D) the final data set is information-clock sampled
microstruture variables for each of two empirical studies.

4.6 Results

Analysis of HMM-based generative feature embeddings

Analysis of generative features from real data may reveal collinear structures in the kernel
classifier. Removing these would improve out-of-sample classification accuracy. To address
collinearity and enhance accuracy, we examine the separability of learned generative features
in real market data [167, 168]. This involves both correlation analysis between feature
embeddings day by day for each asset, as well as visualizations to examine relationships
between features and target labels in multiple dimensions [182].

In this section, we have three objectives, ranking individual and pairs of features by
covariance and applying RadViz and t-SNE for separability analysis. The same features
and labels are used as in the hybrid model of Chapter 5, both with and without kernelization
for RadViz and t-SNE. This reveals links between classification accuracy and separability,
justifying our multi-kernel approach, as linear or single kernel classifiers cannot sufficiently
separate all class pairs based on their geometry [183].
108 4.6. Results

Feature visualisation

This pipeline provides insight into the separability of features and guides the design of
our experimental scheme. Next, we present the results of applying different visualisation
techniques on both the raw and kernel-ised features.

APF.L feature correlation CPG.L feature correlation KGF.L feature correlation

Figure 4.3: Using a 2D feature ranking algorithm, this set of graphs represents Pearson’s correlation
coefficients between features and the dependent variable for symbols APF, CPG, and
KGF for the experiments in Chapter 6. Focusing on transition state embeddings (ξ ), and
the state space embeddings (γ ) and Fisher score derivatives (λ), the analysis improves
decision-making. Fisher score derivatives demonstrate negative correlations, whereas
generative embeddings show mixed results. This visualization helps refine models,
emphasize impactful observations, and helps select features for improved predictive
performance.
4. Data and their statistical transformations
109

Figure 4.4: A one-dimensional feature ranking algorithm assesses each feature individually using
the Shapiro-Wilk test for normality. This evaluation is crucial as it guides the choice
between parametric and non-parametric methods. High Fisher score derivative values
suggest a normal distribution, suitable for parametric methods. In contrast, low values of
ξ and γ values indicate a nonnormal distribution, requiring non-parametric approaches.
Integrating Fisher score derivatives and state-space embedding can provide deeper
insights into HMM generative embedding relationships, informing feature selection.

t-SNE Visualization

We next explore t-SNE visualizations of the kernelized generative features versus the raw
observations. This reveals whether kernelization induces stronger discrimination between
110 4.6. Results

the two classes from our labeling. We project the high-dimensional features into 3D, applying
t-SNE on both raw and RBF-kernelized features. This produces distinct visuals conveying
the rich structure of the features. We utilize the Barnes-Hut t-SNE algorithm for efficient
computation on large datasets.
The contrasting raw and kernelized visuals provide insight on how kernelization improves
visual feature discrimination. Next we describe the steps of the methodology and showcase
experimental t-SNE results across assets and time periods. We will present on three UK-
based companies - KGF, ITV, and APF - spanning diverse sectors. Despite differences, the
commonalities of LSE-listed firms make them suitable for this comparative study.

t-SNE Data Processing We preprocess the data by normalizing features and applying a
polynomial kernel transformation, consistent with the classifier experiments. This ensures
proper scaling for similarity computations.

t-SNE Configuration Key parameters include 3D embedding to represent the 21 fea-

tures, perplexity of 30 to balance the local and global structure, and 1000 iterations. As a
visualization technique, the parameters of the t-SNE are set qualitatively.

t-SNE Results The visualizations reveal clearer separation between classes for kernel-
based versus raw features for certain assets such as APF. This demonstrates the discrimina-
tive power of a simple kernel mapping, motivating its use for classification. Although some
symbols exhibit overlap in the raw scatterplots, kernelized mappings better segregate classes
in many instances. The improved class separation from kernelized t-SNE provides an intuitive
visualization of how the kernel classifier can increase predictive accuracy.

RadViz
4. Data and their statistical transformations
111

Figure 4.5: Figure A: t-SNE visualization of ITV feature evolution across binary labels on 17 November
2017. Left: Raw features exhibit minimal discriminatory capacity despite dimensionality
reduction. Right: Kernelized features (RBF kernel) in 3D space, demonstrating enhanced
inter-cluster differentiation. While t-SNE’s stochastic initialization precludes reliable
inference of relative cluster positions, the kernel-induced manifold elucidates latent
structural properties.

Figure 4.6: CPG.L RadViz for Label 3 Figure 4.7: REL.L RadViz for Label 4

Figure 4.8: KGF.L RadViz for Label 2 Figure 4.9: KGF.L RadViz for Label 3
112 4.6. Results

4.6.1 Parameter values for labels

We introduce six parameterizations that capture a range of market behaviours, from short-
term changes to longer-term trends, using varying lookback windows and thresholds. These
cater to differing trading objectives, risk targets, and market conditions, with the optimal
parameterization dependent on the asset class, trading frequency, and market dynamics.

1. Label One (Binary point in time, Lookback window: 40, Updown threshold:
0.05%): Firstly, we use a long lookback window of 40 trades and a tight threshold of
0.05%. This aims to capture longer-term trends in the market while being sensitive
to small price movements. This combination is suitable for identifying gradual shifts
in market sentiment or detecting slow-moving trends. It may be useful for trading
strategies that aim to capture persistent directional moves.

2. Label Two (Binary point in time, Lookback window: 8, Updown threshold: 0.09%):
With a short lookback window of 8 trades and a slightly higher threshold of 0.09%, this
parameterization focuses on capturing short-term price action. It is more responsive to
immediate changes in the market and can help identify quick price fluctuations or short-
term trading opportunities. This parameterization may be valuable for high-frequency
trading strategies or for traders looking to capitalize on short market inefficiencies.

3. Label Three (Multiclass point in time, Lookback window: 24, Updown threshold:
0.05%): This parameterization uses a medium-sized lookback window of 24 trades
and a tight threshold of 0.05%. The multiclass labeling allows for a more granular
categorization of price movements. It aims to capture intermediate-term trends while
being sensitive to small price changes. This combination can be useful for strategies
that require a balance between short-term responsiveness and medium-term trend
identification, that have position sizing link to market conditions.

4. Label Four (Binary point in time, Lookback window: 9, Updown threshold: 0.41%):
With a relatively short lookback window of 9 periods and a higher threshold of 0.41%,
4. Data and their statistical transformations
113

this parameterization is designed to identify significant short-term price movements. It

focuses on capturing substantial price jumps or drops that may occur due to important
market events, news releases, or sudden shifts in investor sentiment. This parame-
terization can be valuable for event-driven trading strategies or for risk management
purposes, such as setting stop-loss levels.

5. Label Five (Multiclass point in time, Lookback window: 15, Updown threshold:
0.10%): This approach uses a 15-period lookback window and a threshold of 0.10%.
The multiclass labeling provides a detailed categorization of price movements, aiming
to capture short to medium-term dynamics while being moderately sensitive to changes.
This combination can be valuable for strategies that adapt to market conditions or for
traders who want a detailed view of price action.

6. Label Six (Binary point in time, Lookback window: 15, Updown threshold: 0.03%):
With a medium-short lookback window of 15 periods and a very tight threshold of
0.03%, this parameterization is highly sensitive to small price fluctuations. It focuses on
capturing even the slightest price movements over a relatively short time frame. This
combination may be useful for identifying microstructure noise or for trading strategies
that aim to exploit very short-term market inefficiencies, such as high-frequency trading
or market making.

We use a range of options for capturing different aspects of market behavior, from short-term
fluctuations to longer-term trends, and from binary to multiclass categorizations. The choice
of parameterization depends on the specific trading objectives, risk tolerance, and market
conditions of interest.
114 4.6. Results

Label Type Lookback window Updown threshold

One Binary point in time 40 0.05%
Two Binary point in time 8 0.09%
Three Multiclass point in time 24 0.05%
Four Binary point in time 9 0.41%
Five Multiclass point in time 15 0.10%
Six Binary point in time 15 0.03%

Table 4.1: Parameter settings for each label in the experiments. The updown threshold is used to
trigger the labeling mechanism. This table presents the experimental parameter settings
for six different labels (one to six) in the context of financial time-series analysis. Each
row represents a distinct label, and the columns provide information on the label’s type,
look-back window, and updown threshold (expressed as a percentage).

Figure 4.10: These plots show examples of the labeling mechanism using price-threshold differentials.
Positive differentials generate +1 labels (top graph), while negative differentials yield -1
labels (bottom graph). Red dashed lines represent the thresholds determining these
labels.
4. Data and their statistical transformations
115

Figure 4.11: Distribution of proportion Figure 4.12: Distribution of proportion

of negative versus positive of negative versus positive
classes for Label 5. classes for Label 2.

Figure 4.13: Distribution of proportion Figure 4.14: Distribution of proportion

of negative versus positive of negative versus positive
classes for Label 3. classes for Label 1.

4.7 Summary

This chapter establishes the foundation for the statistical methodologies and data transfor-
mation techniques used in the thesis. We discuss the use of trades-and-quotes (TAQ) data
from limit order books, which necessitates statistical and machine learning methods to model
high-frequency processes. The concept of information clocks is introduced to investigate
microstructural dynamics, including volume clock, tick clock, and dollar clock.

We presented a methodology to address the classification challenges associated with

highly imbalanced high-frequency data, utilizing automated labeling techniques based on
dynamic thresholds derived from empirical quantile estimates. We outline a generic data
processing pipeline for converting financial information exchange (FIX) data into feature sets
sampled using information clock techniques.
116 4.7. Summary

Feature visualization methods like t-SNE and RadViz are employed to analyze the
separability of features and guide the experimental design, with results presented from
applying these techniques on both raw and kernel-ized features. This shows how kernelization
can improve visual feature discrimination. Throughout, the importance of feature selection
in handling high-dimensional, noisy data is emphasized. In general, we presented the
groundwork for the data-driven methodologies used in subsequent parts of the thesis, focusing
on capturing intrinsic imbalances and evolving market trends in high-frequency datasets to
support the development of machine learning algorithms for trading applications.
Chapter 5

Generative-discriminative machine learn-

ing models for regime classification

We present a hybrid machine learning approach combining hidden Markov models

(HMM) and support vector machines (SVM/MKL) to classify high-frequency financial
regimes and predict trade directions. The HMM captures temporal dependencies in
financial time series, creating generative feature embeddings which are then used as an
input into the SVM/MKL classifier for predictive discrimination. This method, requiring no
manual feature engineering, consistently outperforms other classifiers across six trading
regimes. Our HMM-SVM-MKL hybrid improves the classification of high-frequency time
series, offering significant benefits for financial applications.

117
118 5.1. Introduction

5.1 Introduction

Intraday price volatility spikes in financial markets challenge accurate regime classification,
an important aspect for trading strategies that exploit short-term anomalies, particularly in
algorithmic execution, market making, and market surveillance [8, 184–186]. Many existing
methods fail to address key factors, such as the information clock [187], high-frequency
dynamics, model adaptability [188–191], and volatility-induced regime changes.

This chapter introduces a hybrid HMM-SVM-MKL learning approach to overcome these

limitations. Our methodology integrates a hidden Markov model with kernel machines (SVM
and MKL) in a two-stage approach. By applying HMM to microstructure time series data, we
identify regimes that conventional models often miss [68]. We then leverage kernel machines,
known for their predictive prowess when given quality features [143, 192], to enhance
classification accuracy. Our approach extracts descriptive features from the HMM [77, 78]
and employs various kernel classifiers [72, 73, 193].

In algorithmic trading, striking a balance between accuracy and computational efficiency

is paramount. Our method achieves this by combining the strengths of generative and
discriminative approaches [72, 194]. We draw inspiration from Zhang et al. [195], who utilize
graph neural network encoders for feature processing and contrast learning for multivariate
time series representation. We demonstrate our model’s efficacy by classifying six distinct
intraday regimes using high-frequency data from 40 major FTSE100 stocks. Our approach
shows improved accuracy over existing methods, advancing the field of regime identification
modelling [8, 196, 197]. Additionally, we explored feature visualizations, including t-SNE and
RadViz, to enhance interpretability, as detailed in Section 4.6.

5.2 Background and hypothesis

The microstructure of financial markets is dominated by the links between the information
flow, the behaviour of the trader, and the price discovery process [198, 199]. Specifically in
the case of algorithmic trading [200], it greatly influences price discovery and volatility and
5. Generative-discriminative machine learning models for regime classifica-
tion
119

has a stronger impact on auto-correlation of trades compared to human trades.

While past studies have offered variant perspectives on this relationship [201], we aim
to bridge conventional market microstructure understanding with machine learning tech-
niques [202]. As such, a core premise of the focus of this work is that the order flow data,
which provides detailed information on the sequence of trades, can be analyzed to examine
the return profile generated by the trading activities of different participants [3, 46].
Murphy et al.[203], show that short-term asset returns are closely linked to the dynamics
of the limit order book, particularly the imbalance of order flow and trade sequences. This
suggestion directly motivates our choice to exploit microstructural patterns.
Inherently linked is the idea that asset returns are more predictable in a high-frequency
trading environment, and the dynamics in the limit order book, can be exploited by algorithms
to achieve better trade executions [204, 205].
Wu et al. [206], discuss that integrating market microstructure model with machine
learning techniques can offer a robust approach to explain and understand complex market
dynamics. The proposed hybrid methodology of this study provides a framework to model
order flow and the price discovery processes, in a setting appropriate for algorithmic trading.

Hypothesis The application analysis will aim to validate the following financial hypotheses:

1. Market participants’ behaviour is driven by the presence of volatility and the arrival of
trades. The detection of such regimes and their utilisation in predictive modelling for
market making and trading applications can be captured by generative embeddings.
These behaviours are driven by the attractiveness of strategies such as rebate trading
or maker-taker strategies, depending on the prevailing market conditions.

2. To test this hypothesis, we employ an HMM-SVM-MKL framework on high-frequency

transaction data. Aldrich in [190] emphasizes the crucial role of temporal clustering and
trade sequences in high-frequency data for short-term price discovery, underscoring the
importance of modeling trading regimes for accurate market predictions. Furthermore,
Murphy [203] demonstrates that these factors are key drivers of market behaviour,
120 5.3. Modelling methodology

significantly impacting short-term volatility fluctuations.

3. The features extracted by the generative HMM embedding can capture and thereby
represent the behaviours of the market participants, which can help uncover effective
trading strategies. For the most part, the HMM embeddings derived can have a financial
interpretation despite having been derived in a structured and automated way.

These hypotheses are formed by building on the literature in this area. In particular,
our work is motivated by the idea that microstructure models should be viewed as learning
models, where traders rely on private or market information [46]. Such learning models
should capture stylized facts about price distributions to characterize market regimes via
state-dependent returns [68]. Trades, reflecting information quality, become central in such
models where market participants make decisions based on new information.

5.3 Modelling methodology

This work builds on recent advances in hybrid learning for sequence modeling [207], inte-
grating modern machine learning techniques for the discriminative element [191]. We use a
two-stage modeling approach.
We note that we opted to consider these methods in favour of alternative approaches
involving deep learning methods for a few reasons. The first is they are open more readily to
direct interpretation. As highlighted by Kearns and Nevmyvaka [4], two-stage models strike
a balance between interpretability and predictive power. The HMM component captures
underlying market dynamics and regime shifts, while the kernel component focuses on
classification performance. In contrast, single-stage models like neural networks may lack
interpretability due to their black-box nature [180, 208].

First stage: choosing a probabilistic model of market microstructure

In the first stage, and influenced by Cartea and Jaimungal [68] and Kearns and Nevmyvaka [4],
we propose a probabilistic model that integrates transaction data with non-linearity and
5. Generative-discriminative machine learning models for regime classifica-
tion
121

heterogeneity. The hidden dynamic structure of executed trade prices and inter-trade arrival
times is inferred, motivated by [68], who focused on joint dynamics and their simultaneous
interdependence. We also test this hypothesis in 5.4.
A central element of this approach is a hidden Markov model (henceforth HMM) that
identifies intraday trading states and their persistence, focusing on trade price revisions
and arrival times, as in [68]. This model can incorporate expert prior knowledge about the
underlying variables; for example, setting the number of hidden states to three—representing
bull, bear, and neutral market regimes—appeals to market intuition, [191].
This type of HMM would be flexible for handling cross-sectional and temporal dependen-
cies in financial data; which makes it a natural choice for modeling these dynamics. However,
it will lead to fewer states than the maximum likelihood approach introduced by Pohle et
al [209].
Therefore, the first stage involves fitting an HMM to each trading day using the previous
day’s data to identify different market regimes. The HMM captures the underlying market
dynamics by modeling the sequential relationships between trades, focusing on trade price
revisions and arrival times [210].
After fitting the model in the first stage, we obtain the probability distribution of trades with
non-zero price and zero-price revisions. One of the most relevant characteristics in this model
is the duration between trades, as it can identify trading behaviour over short periods given
the information contained [68, 190, 211]. The transitions between states in the HMM are
learned from observing sequences of trades to build the transition probability matrix [210].

Generative embeddings We introduce a principled approach to extract features using

the sequential modeling strength of an HMM for flexible representation of data sequences
in conjunction with kernel-based discriminative learning, inspired by Li and Liu [207] and
Bicego [78, 212].
Generative embeddings like Fisher scores and information matrices, derived from HMMs,
capture model structure sensitivity and information content [77, 213, 214]. In our case,
these embeddings extract informative features from trade and quote data for discriminative
122 5.3. Modelling methodology

classifiers [78, 215]. Daily, a new HMM-model is fitted using previous day’s data, generating
initial embeddings that are updated real-time. Fisher scores extract gradients and curva-
ture information from model parameters, capturing underlying sensitivity and information
content. Generative models capture the underlying structure of the data and learn the rela-
tionships between different variables [72]. Order book embeddings are known to improve
classification [58].

Second stage: Kernel classification

The second stage we use the HMM-derived embeddings as input features for kernel classifiers.
Each classifier models boundaries between intraday market regime classes, potentially
indicating future trade directions. Initially, an SVM is trained to classify market movements or
regimes, harnessing the discriminative power of kernel machines. Subsequently, multiple
kernel learning (MKL) approaches address financial data complexities, integrating various
kernel functions [62, 73]. This discriminative learning enhances the generative modelling
output [72, 213, 216], compensating for the HMM’s limited discriminative ability [80].

Multiple kernel learning

The proposed HMM-SVM-MKL learning process is set up in such a manner that it will be
inherently tied to the price distribution and how prices adjust in response to information
flow. This is based on characterizing market regimes with a state-dependent distribution
of logarithmic returns, adjusting for any identifiable activity [68]. These states highlight
the balance between informed traders and noise traders in the price discovery process.
Moreover, a significant implication of this approach to studying microstructure is the potential
to use order flow data to better distinguish the return profiles associated with different trading
activities [3, 46]. This not only offers insights into high-frequency trading dynamics, but also
emphasizes the potential of exploiting limit order book dynamics for algorithmic strategies.

Although the use of single kernel approaches [63–66] provided foundational groundwork,
multi-kernel approaches align more closely with the intricacies of financial data and have
5. Generative-discriminative machine learning models for regime classifica-
tion
123

shown success in similar applications [62, 67]. Our proposed learning approach combines
the benefits of generative and discriminative models.
The generative HMM-SVM-MKL method explored in this work allows for more direct
estimation and calibration using well-established and reliable statistical methods in practice.
Finally, they work for financial high frequency trade and quote data for any market, both
liquid and illiquid markets, on a daily basis. There are existing discussions on the challenges
that deep learning methods face when applied to the types of high frequency financial data
studied in this paper, and their tendency to overfit in such settings.

Design of the experimental flow

A stepwise procedure of the proposed methodology is described next and presented in

Figure 5.1.

1. Accumulate training data: from TAQ data compute returns and inter-arrival durations
based on trade clock.

2. Fit HMM model with 3 states for each day in the sample of length T.

3. Compute HMM-based features for each day i = 1, . . . , T.

4. Fit SVM/MKL models for each day i, and combination of labels 1 to 6 (6 models per
day) using grid search for parameters C and γ . For our purposes, we use polynomial
kernels for MKL and RBF for the single-kernel cases.

5. Evaluate out-of-sample for each day j : j > i, for each combination 1 to 6 (6 models
per day, one for each parametrisation of the label mechanism).
124 5.3. Modelling methodology

Training HMM HMM kernel Test Accuracy

data algo embed model data

Step 1: Train Step 2: Test

Figure 5.1: In Step 1, TAQ data are used to train the HMM algorithm and derive features, which are
then used to train the various kernel-based classifiers. In Step 2, the classifier is applied
to out-of-sample data and the performance is evaluated.

Core methodological ideas

In summary, the core ideas from a modelling perspective are:

1. A principled hybrid approach (generative embeddings used as input into a discriminative

algorithm), which removes the ad hoc choice of handcrafted features.

2. A HMM-induced feature construction method automates the building of features in a

way that is efficient. This can be adapted to a wide range of time-series structures.

3. Specialised time series kernel structures are considered based on Fisher kernels and
other generative embeddings. Explicit and efficient algorithm are developed for the
computation of the Fisher score, the information matrix and other embeddings. The
Fisher score algorithm is the gradient of the log-likelihood function with respect to the
model parameters.

4. In addition, we make use of the automated process for making labels for classifier
training from Section 4.4.

5. Last, we utilise the methodology proposed by Bertolini and Finch [217] to evaluate
feature stability across various market conditions and analyze convergence properties
for generative features.

The performance of the hybrid framework is evaluated in multiple variants, validating its
accuracy and execution time for practical financial applications. Experiments compare the
two-stage model’s performance against single-stage models to validate its effectiveness.
5. Generative-discriminative machine learning models for regime classifica-
tion
125

5.3.1 Market structure captured by hidden Markov models

Analysing high-frequency trading data reveals market dynamics through tick-by-tick trade
durations and price revisions [196]. The joint distribution of these variables exhibits varying
statistical properties across trading regimes, each dominated by participants with distinct
information and payoff functions [196, 218–220]

HMMs model stochastic systems with observable and unobservable state components.
They comprise of a hidden state process Z (system state) and an observation process O
(parametric distribution modelling observations). Two fundamental assumptions: the latent
process is a finite-state Markov chain, and the observation process distribution depends only
on the current hidden state (see Definition in Appendix B).

Definition 5. Hidden Markov model. A hidden Markov model is a couple of discrete-time

stochastic processes (Zi , Oi ), i = 0, . . . , T, where Z is a Markov chain, i.e. a stationary
and homogeneous process that takes values in the finite set of states Z = {ζ1 , . . . , ζK } and
such that

P [Zi+1 = ζ|Z1 , . . . , Zi ] = P [Zi+1 = ζ|Zi ] , (5.1)

for i = 0, . . . , T − 1, ζ ∈ Z;

O is vector-valued and such that at any time ti , i = 0, . . . , T, the distribution of the random
vector Oi conditional on all the information collected up to time ti depends only on the state
of the latent process,

P [Oi |O1 , . . . , Oi−1 , Z1 , . . . , Zi ] = P [Oi |Zi ] . (5.2)

We model the relationship of trade durations and logarithmic returns as two independent,
continuously distributed random variables [68]. An observation process is a two-dimensional
object with a parametric distribution, the exact parameters of which are governed by the states
of a latent Markov chain. We assume that we have a finite number of trades (observations) T,
126 5.3. Modelling methodology

indexed by i = {0, . . . , T}. The latent process is a finite-state Markov chain and, as such, its
dynamics is governed by a K × K transition probability matrix A with elements ajk ∈ [0, 1],

def
ajk = P [Zi+1 = ζk |Zi = ζj ] , (5.3)

i = 0, . . . , T − 1, ζj , ζk ∈ Z.

The distribution of the initial state at t = 0 is π = (π1 , . . . , πk ),

def
πk = P(Z0 = ζk ), ζk ∈ Z. (5.4)

In this work, log denotes the natural logarithm, i.e., the logarithm with base e, where e is
Euler’s number. We denote by P : R × R+ × Z → R the probability function of the complete
data at any time ti . The marginal probability of the observation data (trade durations and
trade log-price revisions, which is known as the emission probability) is, of course, dependent
on the latent process and will be denoted by POi |Zi =ζ , ζ ∈ Z for compactness, i.e.,

def
POi |Zi =ζ (o) = P (Oi ∈ o|Zi = ζ) , (5.5)
def
o = (ξ, τ ) ∈ R × R+ , i = 0, . . . , T.

However, given our earlier assumption on the independence of the distribution of duration
and price revision conditional on a state, we may write

POi |Zi =ζ (o) = PΞi |Zi =ζ (ξ)Pτi |Zi =ζ (τ ). (5.6)

We assume that the marginal distribution of trade durations follows an exponential density,

Pτi |Zi =ζ (τ ) = λζ exp(−λζ τ ), λζ ∈ R+ . (5.7)

We model the trades for a selection of stocks listed on the FTSE100, where on any
given calendar date we treat each day as a separate data set comprised of intra-daily data,
5. Generative-discriminative machine learning models for regime classifica-
tion
127

Figure 5.2: Logarithmic duration in milliseconds (on the y axis) versus the relative traded price return
(on the x axis) for Anglo Pacific Group, observed over two trading days, 4 August 2017
(left) and 26 September 2017 (right). There is a cluster around trades that arrive quickly
and cause either a very small or a very large perturbation of price. As the duration
increases, the price impact is more meaningful.

producing one fitted HMM per day per symbol. We denote the event times for these trades
by {t1 , . . . , tT }. These times are recorded as the trades occur and need not be equally
spaced; they form event-spaced time series data. The duration between two consecutive
def
trades is denoted τi = ∆ti = ti − ti−1 , τi ∈ R+ . The price process of the symbol we
are studying, sampled at the time of the symbol’s trades, that is, the price of the i-th trade,
 
def
is Pi , i = 0, . . . , T. Logarithmic returns are ξi = log PPi−1
i
, i = 1, . . . , T. For these
observation processes, we will interchangeably use the notation Oi or (ξi , τi ). Given a
discrete-time stochastic process, for example, the hidden state process Z, we will use the
compact notation Zk:l to denote the collection of values {Zk , Zk+1 , . . . , Zl }, k < l. A similar
notation applies to any other process. Lastly, we will denote the realization of a stochastic
def
process along a given path with lowercase letters, e.g. zk:l = {zk , zk+1 , . . . , zl }.

Remark 1. Notice that the assumption of independence in the distribution of duration and
price changes conditional on the latent process, within the context of an HMM, does not
128 5.3. Modelling methodology

contradict the intuition offered to us by inspecting the data in Figure 5.2, that is, a dependence
structure between duration and price changes, since from the perspective of our model, this
is only the marginal density, once the latent process has been integrated out.

Remark 2. The choice of the probability density for trade durations differs from that used
in [68], as the exact format of the exponential distribution without quantisation is utilised.
There are well-established degeneracies in any high-frequency financial data set that would
justify a quantisation approach, particularly when one is concerned with modelling the limit
order book; see Filiminov and Sornette [221, Section 4] for a detailed discussion on the
‘bundling’ effect of high-frequency data and plausible explanations. However, as we are
only concerned with the trades data on any given symbol and having access to data with
microsecond accuracy, we found no significant differences when using quantisation for the
trades duration density, and hence chose to work with the exact density.

A12 A23

ζ1 ζ2 ζ3

O1 O2 O3

Figure 5.3: A HMM with states ζi that emit variables Oi , i = 1, 2, 3. The transition probability between
two states ζi and ζj is Aij , Equation (5.3). This example shows a one-step process in
which states can only reach the adjacent state or themselves.

Log-price revisions are modeled according to a mixture distribution that arises naturally
from the trade data. We often see a significant number of trades that occur at the same price,
i.e., at zero price revision. In contrast, we model the nonzero part of the price revision as
a normal random variable with zero mean and standard deviation that again depends on
the state of the latent process. Specifically, the probability density function of the log-price
revision for our trades data set is

def
fΞi |Zi =ζ (ξ) = wζ δ(ξ) + (1 − wζ )fN (ξ; 0, σζ ) (5.8)
5. Generative-discriminative machine learning models for regime classifica-
tion
129

where

def
1. wζ = P(ξi = 0|Zi = ζ) is the conditional probability of observing a zero price revision
trade given the event that the hidden process is at ζ .

2. δ(ξ) is the Dirac delta function centered on zero.

3. fN (ξ; 0, σζ ) is the probability density of a normal random variable with mean 0 and
standard deviation σζ .

To be able to do any meaningful work with our parametric model, one needs to learn/calibrate
all parameters to the data. Therefore, to facilitate discussion, we will denote the set of all
def
parameters that define our discrete state HMM as θ = {π, A, λ̄, w̄, σ̄}, λ̄, w̄, σ̄ ∈ RK ,
where, for example, λ̄ is the vector of trade arrival parameters with entries K , one for each
hidden state and similarly for the other two vectors. The density of the complete data and the
marginal density of the observation data will depend on the vector θ . When we need to be
explicit about the density dependence of this vector, we will use the notation P(o; θ). The
standard approach to calibrating HMMs is the expectation maximisation algorithm that we
describe briefly here and provide the algorithm in detail in the Appendix B.

5.4 Data

The data consists of millisecond resolution quotes and trades for 40 FTSE 100 stocks over
the 2017 calendar year. In particular, the time stamps feature millisecond precision, with an
accuracy of 10−7 and sourced from Reuters. The raw data includes trades and (top of the
order book) quotes (TAQ) and OHLC information. The key variables extracted include time
stamps, trade prices, bid/ask, and quoted volumes. This allows modeling of microstructure
dynamics, such as inter-trade durations and price changes, on an intra-day basis. To
construct the features, the models were fitted separately for each symbol and day to generate
embedding features. Kernel classifiers are then built on top of these features. Table 5.1
compiles an aggregate summary, presenting an overview of the 40 assets throughout all
130 5.4. Data

trading days of 2017. Summary statistics like mean duration, number of trades, return
volatility, and traded volume in exhibit high variability between different stocks. The data
exhibit a wide range of market microstructure dynamics. Modeling this heterogeneity is a key
motivation for the generative embedding approach. This can be seen from the fact that for
instance the mean duration ranges from 0.41 (CCL.L) to 1.29 (AV.L), while the mean number
of trades varies from 789.5 (APF.L) to 5493.0 (BLT.L). The mean return standard deviation
spans from 0.40 (SDR.L) to 0.55 (AZN.L), and the mean traded volume remains relatively
consistent at 900.00 for most stocks, with a few exceptions, such as 509.00 (BLT.L) and
500.00 (CCL.L).

Metric Median
Duration/milliseconds 0.71
Standard deviation of the log return 0.46
Number of trades in a day 2115
Number of traded shares 717.25

Table 5.1: Summary of data for all 40 symbols for all trading days of 2017. Duration is the difference
between the calendar time of sequential trades. The table summarizes key trading statistics,
such as the duration in milliseconds, the standard deviation of logarithmic return, the
number of trades per day, and number of traded shares.

Choice of symbols to illustrate the methodology

To analyze high-frequency trading dynamics across different market sectors, we focus on

five representative FTSE 100 stocks: APF.L, CCL.L, CPG.L, RBS.L, and KGF.L. These
stocks span key sectors such as basic materials, consumer goods, banking, and retail.
Looking at stocks with diverse properties helps to evaluate the effectiveness with which the
proposed framework models various microstructure regimes. For example, CCL.L shows
rapid dynamics with an average duration of 0.41 seconds between trades, while APF.L is
slower at 0.56 seconds.
Examining assets with differing liquidity, volatility, and other characteristics provides
insight into how the sector influences high-frequency strategies through feature engineering,
prediction accuracy, and trading performance [200]. Despite concentrating our analysis on
5. Generative-discriminative machine learning models for regime classifica-
tion
131

these stocks, we also include features from other FTSE 100 symbols for a comprehensive
perspective. In general, this cross-sector diversity offers a valuable understanding of the
complexity of high-frequency trading in the real world.

Stock Mean Mean nr. Mean stdev Mean

ticker duration of trades of returns traded volume
AAL.L 0.50 845 0.53 900.00
APF.L 0.56 790 0.55 900.00
AV.L 1.29 845 0.53 900.00
AZN.L 0.58 1487 0.55 900.00
BARC.L 0.75 1778 0.54 900.00
BATS.L 0.54 4488 0.48 900.00
BLT.L 0.48 5493 0.43 509.00
CCL.L 0.41 4576 0.46 500.00
CEY.L 0.49 2371 0.49 510.00
CPG.L 0.46 1768 0.46 600.00
CPI.L 0.49 1688 0.49 679.25
ITV.L 0.53 1653 0.50 717.25
KGF.L 0.50 1713 0.48 750.00
LAND.L 0.50 1652 0.48 720.50
LGEN.L 0.54 1713 0.47 750.00
LLOY.L 0.60 1780 0.46 900.00
MAB.L 0.70 1717 0.47 750.00
MKS.L 0.76 1780 0.47 900.00
NG.L 0.78 1934 0.46 753.50
PRU.L 0.75 2153 0.45 750.00
PSON.L 0.71 2082 0.46 752.62
RB.L 0.71 2163 0.45 750.00
RBS.L 0.72 2193 0.45 753.50
RDSa.L 0.71 2260 0.44 748.50
RDSb.L 0.73 2314 0.43 695.75
REL.L 0.71 2284 0.41 674.62
RR.L 0.69 2209 0.42 700.00
RTO.L 0.71 2188 0.41 706.50
RSA.L 0.51 811 0.50 900.00
SDR.L 0.72 2165 0.40 700.00
SGE.L 0.71 2139 0.41 706.50
SHP.L 0.69 2165 0.42 700.00
SMIN.L 0.71 2115 0.41 674.62
SPT.L 0.73 1883 0.42 677.00
STAN.L 0.75 2115 0.41 663.00
TSCO.L 0.76 2165 0.41 676.00
ULVR.L 0.74 2198 0.40 659.75
UU.L 0.77 2165 0.41 641.00
VOD.L 0.74 2198 0.40 661.00

Table 5.2: Mean duration, number of trades, mean return standard deviation of logarithmic price
change, and mean traded volume for each stock. All data are from 2017. All symbols were
used in the experiments, but only a subset of the results are presented in this main body.
132 5.4. Data

Examining the assumption of linear independence

We examine the statistical assumption of non-linearity between inter-arrival times and micro
price changes across multiple trading symbols for our data set, on an intraday basis. Pearson
correlation coefficients are computed, (a robust tool for measuring linear associations) and
conducted hypothesis testing. Our analysis consistently led us to fail to reject the null
hypothesis, suggesting no significant linear relationship between these variables across the
dataset. This outcome, indicated by a high median p-value, reinforces the assumption that
inter-arrival times and micro price changes can be modeled as independent features in this
high-frequency setting.

However, we should caveat that the relationship is nonlinear (as shown in the cluster
of 5.2), the Pearson coefficient may be close to zero (thus not capturing such complexities),
indicating no linear correlation even though a relationship exists. This topic will be investigated
in subsequent work.

For each symbol, a hypothesis test is conducted to assess the statistical significance of
the observed correlation. The hypotheses are formulated as follows:

H0 : ρ = 0 (no correlation exists between the variables),

H1 : ρ ̸= 0 (a non-zero correlation exists between the variables).

Where ρ represents the population correlation coefficient. The test statistic r is used to
compute a p-value, which evaluates the probability of observing the data if the null hypothesis
H0 were true.

This independence validates (and simplifies) the modeling of their joint distribution,
supporting the use of models that treat these variables as separate inputs, which may
enhance the robustness and simplicity of algorithmic trading strategies.
5. Generative-discriminative machine learning models for regime classifica-
tion
133

Figure 5.4: The overall median correlation coefficient is -0.0018, indicating a negligible linear relation-
ship between inter-arrival times and micro price changes. With an overall median p-value
of 0.0383, the analysis across all symbols leads to a failure to reject the null hypothesis,
affirming the assumption of no significant linear relationship. These findings suggest that
the joint distribution of arrival rates and micro price returns can, in this setting, be modeled
as having no linear correlation, simplifying the complexity of the market structure model.

Investigation of feature properties via simulation study

Understanding the statistical properties of features that are inputs for high-frequency algo-
rithmic trading applications is critical for performance understanding and improvement. We
examine the convergence properties of the features via simulation.

5.4.1 Examining feature embedding stability and empirical characteristics

First, we examine how the estimation of the generative features changes as the length of the
data sequence used to make the embedding features increases. As the sample size grows,
we want consistency in the estimated feature sets. This indicates that the features generated
are stable when enough data is used in their construction.

To investigate this, we begin with a synthetic data study that employs data generated
134 5.4. Data

from a fixed HMM model based on parameters obtained from calibrations to a real data set.
As a result, we look at several synthetic time series generated by Monte Carlo simulation of
the underlying distribution, which is initialized to the symbol LLOY.L. This ensures that the
synthetic experiments are based on realistic parameter ranges for relevant intertrade duration
and price distributions. In this stylized example, the number of states is set to two (hereafter,
k = 2).

Therefore, the values for the model parameters are based on the calibration of the real
data to [Link]. We chose parameters that are sufficiently apart; and initialize the transition
matrix in this experiment to be skewed in favor of one of the two states:
 
0.4 0.6
A= . (5.9)
0.7 0.3

Furthermore, the observation data are generated in such a way that the ratio of latent states
remains constant. These simulation settings were used to produce synthetic time series that
were subsequently used to construct estimates of the generative embedding features. Other
parameter of the model are

π = (0.45, 0.55), λ̄ = (1/3, 1/20) (5.10)

w̄ = (0.7, 0.3), σ̄ = (0.5, 0.002).

We subsequently generate 10,000 realizations (via Monte Carlo simulation) of state

sequences of a fixed length of 4,000 time steps; each state vector at each time step is a
two-dimensional vector corresponding to price change and duration. The length of 4,000 is
selected because it is approximately the average number of trades for the assets under study
daily in 2017. Then, for every sequence of states, we generated the sequence of observation
data from the HMM observation model (again via simulation).

By examining the data, we can demonstrate how the asymptotic behaviour of the gener-
ative features as sample size does. It is important to verify consistency (in the generative
5. Generative-discriminative machine learning models for regime classifica-
tion
135

embedding features) for a non-misspecified setting for very large sample sizes. In order to
achieve this, we first investigate their stability and consistency.

5.4.2 Methodology

This section addresses one of the most important questions, which is how stable the features
are as the initial-state Markov chain approaches equilibrium.

• We first investigate the stability of the generative features by calculating the individual
estimates of the generative embedding at each time point t for each of the sampled
trajectories, each of which has a length of 10,000.

• This is computed for each of the entire set of our generative features (state-space
embeddings and emission-space embeddings, Fisher scores, and information-matrix
embeddings). This gives each of the actualized trajectories a set of embeddings that
change over time. We generate the features for each state sequence and for each of
the synthetic realizations.

• We can then accumulate these features over an increasing number of windows, with
window one having T1 samples, window two having T2 samples, etc., until Tn = 4, 000.

• The eigenvalues of the matrix-valued features are computed by singular value de-
composition. We estimate the spectral gap, the trace, and the determinant of each
matrix-valued feature. We use these stability measures as the sample size increases.
The spectral gap is approximated using the difference between the two largest eigen-
values. The trace is computed by adding the eigenvalues.

• For the vector-valued features, we compute the polar coordinates.

1 P τi t
• We compute the sequence of quantities over the increasing widows through τi t=1 Mi ,

where Mtt represents the spectral gap, the determinant or the trace in each case. The
results were consistent in each summary measure, so we present the results for the
spectral gap studies in Figure 5.5.
136 5.5. Experimental results for kernel classifiers

Figure 5.5: The left image shows the spectral gap evolution of ξ in fixed-length sequences, while
the right displays trace evolution. ξ represents transitions between specified states in
observation sequences. As sequence length increases, the spectral gap converges. This
gap establishes a bound on the Markov chain’s mixing time. The trace plot visualizes the
chain’s longitudinal behaviour.

The simulation study illustrates desirable properties. Small spectral gap values indicate
significant metastable eigenvectors, positive for one state and negative for another. Metasta-
bility relates to the spectral properties of transfer operators in Markov chain dynamics. This
method shows a convergence speed to equilibrium as sequence size increases, with faster-
than-exponential asymptotic convergence. The a priori variance selection allows sufficient
state changes, preventing features from getting stuck in log-likelihood distribution extremes.

5.5 Experimental results for kernel classifiers

Description of experiments Next, we describe a series of experiments aimed at evaluating

the effectiveness of multiple kernel learning in predicting financial market trends and trade
direction using the HMM-generative embeddings.

We first establish two benchmark models - logistic regression and a simple feedforward
neural network - to assess the incremental predictive power gained from using more flexible
machine learning methods capable of handling a larger number of predictors and capturing
nonlinear interactions.

We then move on to applying single-kernel support vector machines to predict trade direc-
tion, examining the impact of label distribution on binary and multiclass efficacy. Additionally,
5. Generative-discriminative machine learning models for regime classifica-
tion
137

we present the experimental results from kernel target alignment and multiple kernel learning
approaches, which utilize the characteristics of each symbol dataset to generate multiple
kernels.

By fitting and training on each day followed by applying the learned classifier out-of-
sample, we obtain a robust assessment of model performance that is more influenced by
kernel quality than classifier choice.

5.5.1 Benchmark experiments

In this part, we describe the results of two benchmark experiments designed for establishing
a comparative framework. These baseline assessments enable us to quantify the enhanced
predictive capacity achieved through the implementation of more advanced algorithms.
Muliple kernel lerarning can accommodate a broader array of predictors and non-linear
interactions, in contrast with the constraints of parsimonious linear models. The superior
performance relative to these benchmarks underscores the potential efficacy of the proposed
modeling methodology.

Market features for the benchmarks

We construct sets of features, based on the following well known indicators the Moving
Average Spread, On-Balance Volume (OBV), and the Chaikin Money Flow (CMF). The
set of moving average spreads identifies trends based on price momentum over different
time frames. It can signal potential reversals when short-term price movements deviate
significantly from a more long term trend. The OBV adds a volume dimension to these
set of features, helping confirm the strength of the detected trend. For instance, if the
price is trending upward and OBV is also rising, it suggests that there is significant buying
pressure supporting the trend. CMF confirms trend strength by measuring volume-weighted
accumulation and distribution over a specified period. A positive CMF during an uptrend
indicates buying pressure, while a negative CMF during a downtrend signals selling pressure.
138 5.5. Experimental results for kernel classifiers

Do the generative features add value?

The initial comparison involves testing on an unknown testing subset after evaluating the
performance of generative embeddings and market features in a logistic regression model
using a training subset. This comparison aims to determine which type of feature (generative
embeddings or market features) performs better in predicting the outcome variable in a
logistic regression model. We find that generative embeddings outperform market features
in predicting the outcome variable in a logistic regression model based on their higher
accuracy and lower error rates. We present the results for 13 symbols. We calculate various
performance metrics (such as the classification report, which provides precision, recall,
and f1 scores and the overall accuracy). This specific experiment includes splitting the

Symbol Accuracy Precision Recall f1 -weighted f1 -micro f1 -macro Hamming loss

AAL.L 0.03 0.21 0.07 0.08 0.03 0.12 -0.03
APF.L 0.02 0.06 0.03 0.04 0.02 0.04 -0.02
AV.L 0.01 0.20 0.04 0.06 0.01 0.08 -0.01
AZN.L 0.03 0.27 0.09 0.07 0.03 0.13 -0.03
BARC.L 0.00 0.18 0.02 0.03 0.00 0.05 0.00
BATS.L 0.06 0.26 0.10 0.09 0.06 0.14 -0.06
BLT.L 0.05 0.25 0.11 0.09 0.05 0.15 -0.05
CCL.L 0.09 0.36 0.23 0.14 0.09 0.28 -0.09
CEY.L 0.02 0.18 0.07 0.08 0.02 0.11 -0.02
CPG.L 0.05 0.28 0.15 0.12 0.05 0.20 -0.05
ITV.L 0.01 0.15 0.04 0.05 0.01 0.07 -0.01
KGF.L 0.02 0.23 0.09 0.06 0.02 0.14 -0.02

Table 5.3: The table lists 13 symbols. Differences between generative embeddings and market
features across key metrics (generative-market) are presented. Positive values indicate
that generative embeddings outperform market features, while negative values indicate the
opposite. For Hamming loss, negative values are favourable since a lower Hamming loss
indicates fewer prediction errors.

dataset into training and testing subsets for model training and evaluation. After training
on the training subset, the logistic regression model undergoes performance evaluation on
the testing subset. Similarly, to the rest of the experiments, this assessment comprises
calculating various performance metrics such as precision, recall, f1 scores, and overall
accuracy in the classification report. In this comparison of two types of features, the first type
is generative embeddings, and the second type is market features.
5. Generative-discriminative machine learning models for regime classifica-
tion
139

Specifically, generative embeddings demonstrate higher accuracy (e.g., AAL.L: 0.71 vs.
0.68), F1 macro (e.g., APF.L: 0.63 vs. 0.59), F1 micro (e.g., CPG.L: 0.72 vs. 0.67), and F1
weighted scores (e.g., KGF.L: 0.75 vs. 0.69), along with superior precision and recall (e.g.,
AZN.L: precision 0.60 vs. 0.34, recall 0.51 vs. 0.42) and lower Hamming loss (e.g., BATS.L:
0.19 vs. 0.25). These findings suggest that the generative embeddings are more reliable and
effective, providing a more balanced prediction performance and fewer errors.

Logistic classification

In the first baseline experiment, we use logistic classification. A 0.5 decision threshold is
used for binary classification, balancing sensitivity and specificity. In multiclass scenarios, the
model was adapted using the one-vs.-rest approach. Each class was considered separately
against all others, with a separate logistic regression model fitted for each. A 0.5 decision
threshold was maintained for each model in the OvR scheme. This approach simplifies the
extension of binary logistic models to multiclass problems, treating each class prediction as
a separate binary classification, and maintaining uniform decision-making criteria across
classes.

In the extension to multiclass classification scenarios, the logistic regression framework

was adapted using the one-vs-rest approach. Each class is considered separately against
all other classes grouped together as the negative class, and a separate logistic regression
model is fitted for each class. As in the binary case, a decision threshold of 0.5 is used for
each model in the OvR scheme, meaning that an observation is predicted to belong to a
particular class if the computed probability from its corresponding model exceeds 0.5. This
simplifies extending the binary logistic model to multiclass problems by treating each class
prediction as a separate binary classification and maintaining uniformity in decision-making
criteria across different classes.

The model’s performance varies across the six labels. The accuracy ranges from 31.32%
to 48.01%, with Label Four having the highest accuracy and Label Six the lowest. Precision,
recall, and f1 scores also vary widely across the labels. Hamming loss ranges from 51.99%
140 5.5. Experimental results for kernel classifiers

to 68.68%, with Label Four having the lowest loss and Label Six the highest.

Feed forward network

Our choice for the second benchmark, is motivated by the work in [222], [223] and [224] who
demonstrate that these methods are largely specialized for prediction tasks, and thus ideally
suited to our problem. We use a sequential model with input layer adapted to the feature size,
two densely connected hidden layers with 64 and 32 neurons, and a sigmoid output layer for
binary classification.

Rectified linear units are used as activation functions for the hidden layers to introduce
non-linearity. The Adam optimizer is chosen due to its adaptive learning rate capabilities,
which are suitable for data with fluctuating statistical properties. The validation involves
minimizing a binary cross-entropy loss function through epochs. This benchmark is a shift
towards a more sophisticated framework capable of capturing complex relationships.

The model has varying performance across different labels, with notable challenges in
achieving high precision. Despite a high accuracy of 97.24% and an f1-weighted score of
95.88% for Label One, the precision is low at 48.62%, indicating frequent misclassification of
non-relevant cases. Similar issues are observed across other labels, particularly in Label Six
where the accuracy drops to 9.54%, accompanied by minimal f1-micro and macro scores
as well as a high Hamming loss of 90.46%. Similarly to some of the other models in our
experiments, it struggles significantly with this label, which implies difficulties with the class
imbalances within the dataset.

5.5.2 Experiments comparing kernel classification algorithms

Next we examine the effectiveness of single-kernel SVMs, MKL and KTA with RBF and
polynomial kernels in predicting trade direction across binary and multiclass scenarios. The
experimental setup includes calculating features intraday for each day, fitting multiple models,
and optimizing parameters using a comprehensive 5-fold cross-validation process.
5. Generative-discriminative machine learning models for regime classifica-
tion
141

Performance metrics to assess HMM-SVM-MKL performance

In the machine learning literature, a classifier is evaluated with precision (positive predictive
value) and recall (sensitivity), which we define below [225, 226]. No single metric fully
captures the efficacy of the model [227], but together these metrics are well suited to evaluate
multiclass imbalance problems. Our goal is to create a pragmatic and practical framework,
the experimental assessment becomes an integral part of our efforts. However, there is no
precise recipe for the evaluation of classification algorithms [225].

1. Precision is the ratio of true positives to the sum of true positives and false positives.
Similarly, the recall gauges the model’s ability to correctly identify true positives.

2. Beyond these individual metrics, the f1 -score offers a harmonized evaluation criterion,
taking into account the trade-offs between precision and recall. To accommodate the
complexity of multiclassification problems, variations such as the macro-f1 , micro-f1 ,
and the weighted f1 score are employed. The latter, in particular, modifies the macro
version to weight out label imbalance.

3. In multilabel classification scenarios, the Hamming loss emerges as an important metric.

It accounts for the number of instances-label pairs that are misclassified, providing a
single comprehensive measure of performance across multiple labels [228].

Single kernel experiments (SVM)

We apply single-kernel SVMs to predict trade direction, examining the influence of label
distribution on binary and multiclass efficacy. The basic SVM model is binary and can only
differentiate between two classes.
For each trading day, we compute features and then fit 6 single-kernel SVMs using RBF
and polynomial kernels given in Section 3.2. We use 5-fold cross-validation to select optimal
C and α values [229]. The cross-validation search is exhaustive in that it iterates over all
possible permutations. For the RBF kernel, the cross-validation space varies in the ranges
C = {10−1 , 1, 10, 102 } and α = {10−5 , 10−4 , 10−3 , 10−2 , 10−1 }. For polynomial kernels, we
142 5.5. Experimental results for kernel classifiers

chose to use homogeneous polynomials, setting the offset to zero, and the degree ν takes
values in the range {1, . . . , 11}.

Again, for the extension to the multiclass problem, we use the monolithic optimisation
approach and build a single model to differentiate different classes. This kernel framework
incorporates many elements from the limit order book and the features induced by the HMM
model that characterize the microstructure.

Kernel target alignment and multi kernel

For the MKL and KTA approaches, we generate a number of kernels for each of the symbol
datasets. Each symbol has different characteristics (volume traded, average arrival of trades,
trade price volatility of the underlying instrument). Our approach of fitting and training on
each day and applying the learned classifier out of sample has the benefit of using a relatively
small training set to obtain the final accuracy of the model. Hence, the results should be
more affected by the quality of the kernel than by the choice of the classifier (to which the
combined kernel methods are applied).

Performance of HMM-SVM-MKL kernel weighting approaches

We test the performance of all algorithms primarily with precision and accuracy, but we also
discuss additional metrics. The results for selected FTSE 100 stock datasets are shown
below. The findings for the KTA and MKL techniques are the median for all test dates in all
performance measures. Since the Hamming loss (HL) is built for multiclass while precision,
recall, and f1 measure are developed for a binary class problem, it is preferable to compare
accuracy with the latter. Using HL, we can see that it is the highest for label 3, but the
lowest for label 1, which corresponds to the highest accuracy metrics for both KTA and
MKL. Table 5.4 summarizes the results which clearly demonstrate the superiority of the
MKL method compared to single-kernel and KTA approaches for this prediction task. Using
multiple kernels, the model is able to capture the intricacies and non-linearities in the data,
leading to enhanced performance across evaluation metrics.
5. Generative-discriminative machine learning models for regime classifica-
tion
143

Model Accuracy Precision Recall f1 -weighted f1 -micro f1 -macro Hamming loss

Logistic 0.41 0.49 0.42 0.47 0.41 0.35 0.59
FFN 0.64 0.32 0.42 0.55 0.64 0.32 0.36
KTA 0.55 0.62 0.55 0.56 0.55 0.38 0.45
MKL 0.79 0.70 0.79 0.73 0.79 0.43 0.21

Table 5.4: Performance metrics for various classification models, showing accuracy, precision, recall,
and f1 scores, along with Hamming loss. From the top row, we start with the logistic
classification benchmark, then the feed forward network and finally the two multi-kernel
implementations.

Figure 5.6: Classification performance for six labels using four methods: logistic classification, feed-
forward neural network, kernel target alignment, and multi kernel learning. The accuracy
and precision metrics are depicted for each label. Accuracy is shown with solid bars,
while Precision is indicated by diagonally-hatched bars. Each method is represented by a
specific color: Logistic Regression in light blue, FFN in green , KTA in pink, and MKL in
dark blue. The y-axis represents the score range from 0 to 1 (0% to 100%), allowing for a
clear comparison of model effectiveness across the labels.

Specifically, the MKL model shows lower Hamming loss and higher accuracy, f1 -macro,
f1 -micro, f1 -weighted, precision, and recall scores compared to the single-kernel method, as
seen in Tables 1 & 2. This confirms the value of combining multiple kernels to represent the
complexity of the data patterns. While the KTA approach also provides competitive results,
144 5.5. Experimental results for kernel classifiers

the MKL method consistently outperforms it on most metrics, indicating it is particularly well
suited for this problem. However, given the dynamic, complex, and nonlinear nature of the
data characterized by regime shifts over time, it is essential to utilize a model capable of
effectively capturing these intricate patterns and transitions. The results show that the MKL
method achieves this by using multiple kernels, providing traders with superior signals for
decision making in fast-moving markets.

5.5.3 Comparing across different labels/regimes

Building on the previous set of experiments, we perform further experiments for the HMM-
SVM-MKL framework with individual kernel based performance weighting in the kernel
mixture (termed the MKL method) versus the MKL method with weightings based on the
KTA approach (termed the KTA method). Results are presented across all labels/regimes
as seen in Tables 5.5 and 5.6, the MKL approach generally again outperforms KTA in key
classification metrics for all labels. The only exception is the f1 -macro score for labels 3, 5
and 6, where KTA shows marginally better performance. However, given MKL’s consistently
strong advantages across the majority of metrics, these results reinforce that MKL is more
capable of effectively combining multiple kernels to enhance the classification of HMM-learned
generative embeddings.

Label Accuracy Precision Recall f1 -weighted f1 -micro f1 -macro Hamming loss

1 0.98 0.96 0.98 0.97 0.98 0.49 0.02
2 0.91 0.84 0.91 0.87 0.91 0.48 0.09
3 0.50 0.50 0.50 0.46 0.50 0.33 0.50
4 0.84 0.70 0.84 0.77 0.84 0.46 0.16
5 0.79 0.62 0.79 0.69 0.79 0.44 0.21
6 0.73 0.55 0.73 0.61 0.73 0.39 0.27

Table 5.5: Results for the MKL algorithm’s performance across all labels and symbols, with metrics
averaged for out-of-sample results.

Logistic classification The accuracy, precision, recall, and f1 scores are generally low
across most labels. Label One’s precision is particularly poor at only 2.62%, and there are
5. Generative-discriminative machine learning models for regime classifica-
tion
145

Label Accuracy Precision Recall f1 -weighted f1 -micro f1 -macro Hamming loss

1 0.81 0.96 0.81 0.85 0.81 0.47 0.19
2 0.71 0.74 0.71 0.72 0.72 0.47 0.29
3 0.33 0.40 0.33 0.32 0.33 0.27 0.68
4 0.71 0.74 0.71 0.72 0.72 0.47 0.29
5 0.33 0.40 0.33 0.32 0.33 0.27 0.68
6 0.42 0.47 0.42 0.40 0.42 0.30 0.58

Table 5.6: KTA algorithm’s performance across all labels and symbols, with metrics averaged for
out-of-sample results.

generally high Hamming losses. This suggests that logistic regression may struggle with the
complexity or imbalance inherent in the dataset.

Feed-forward neural network This approach demonstrates a notable increase in accuracy

and f1 -weighted scores for labels One and Two, suggesting effective handling of these
classes. However, it struggles with label Six, indicating difficulties in managing specific
classes.

Kernel target alignment (KTA) The KTA shows mediocre performance with some consis-
tency across different labels but does not excel in any specific metric. It generally performs
better than logistic regression but still falls behind the feed forward approach in handling the
first two labels effectively.

Multikernel learning (MKL) MKL consistently delivers the highest accuracy and f1 scores
for all labels, especially excelling in One and Two with near-perfect scores. It showcases
robustness and adaptability across various labels, demonstrating significantly lower Hamming
loss, indicating fewer misclassifications. Comparisons against benchmarks and alternatives
validate MKL’s superiority in the HMM-SVM-MKL approach across labels and regimes,
indicating potential for real-world high-frequency or streaming data applications.
Table 5.6 presents the HMM-SVM-MKL version in which mixture weights are formed based
on KTA. The HMM-SVM-MKL with KTA-based mixture weights shows variable performance.
It achieves peak accuracies of 0.812 and 0.715 on binary labels 1 and 4 (40 and 9-period
146 5.5. Experimental results for kernel classifiers

look-back, 0.05% and 0.4% UpDown thresholds). However, for multiclass labels 3 and 5 (24
and 15-period look-back), performance drops significantly: 0.325 accuracy, 0.395 precision,
0.325 recall. KTA struggles with multiclass labeling, limiting its utility for complex applications
such as market making. In contrast, MKL demonstrates consistent strong performance
across all scenarios, underscoring its versatility.

Comparison versus the benchmarks While logistic regression offers faster computation
and simpler implementation, it does not match the performance levels of more advanced
methodologies like multiple kernel learning and kernel target alignment. In our experiments,
logistic regression, despite its operational advantages, is insufficient in accuracy and the
ability to handle the complexities of high-frequency trading data.
For instance, MKL achieved a Hamming loss of only 0.02 and an f1 -macro score of 0.49,
significantly better than logistic regression’s performance on similar metrics affirming the
need for more sophisticated approaches in environments demanding high precision and
adaptability. Also, FFN does provide competent baseline results under certain conditions,
MKL’s superior adaptability, consistency, and overall higher performance metrics make it a
more robust choice for complex, high-stakes environments such as high-frequency trading.
We focus on two particular advantages of MKL:

• Adaptability: MKL’s ability to leverage multiple kernels allows it to adapt dynamically

to each problem by tailoring its approach to the dataset features. This is in contrast
to FFN, which, although capable of learning nonlinear relationships, may not adjust
dynamically based on the characteristics within our datasets.

• Robustness To Complex Data: MKL is better suited for datasets with a wide range of
behaviours and class complexities - crucial for high-frequency trading environments.
The performance dip of FFN in complex labels suggests underfitting or an inability to
generalize well between different data in our study.

Overall, these results demonstrate that the proposed framework, which utilizes individual
kernel performance for the mixture kernel weightings, is an effective dynamic model for
5. Generative-discriminative machine learning models for regime classifica-
tion
147

predicting intraday activity. It effectively addresses key microstructure characteristics such

as time series dependence, periodicity, and asymmetry. The variable model performance
highlights the importance of selecting a suitable approach to handle high-frequency data
complexity. By employing multiple kernels, MKL proves to be a superior methodology for
predicting and categorizing important intraday activities.

5.6 Summary

Our work presents a hybrid machine learning methodology (HMM-SVM-MKL) that com-
bines multiple kernel learning (MKL), hidden Markov model (HMM)-induced features, and
an automated flexible labeling algorithm for advanced financial time series classification.
We demonstrate its effectiveness in classifying trade directionality and short-term high-
frequency regimes for FTSE 100 stocks. This combination is unique and marks a significant
improvement for algorithmic trading and high-frequency time series applications by improv-
ing prediction accuracy and capturing complex market dynamics. A key advantage of our
approach is the ability to handle various types of financial data, such as traded prices, vol-
umes, and intertrade durations, which can be beneficial for modeling state-dependent trading.
The HMM-based Fisher score algorithm enables robust parameter estimation to accurately
characterize market regimes. The derived generative embeddings leverage distributional
changes in prices and durations to identify regimes linked to behaviours of different market
actors. This provides valuable information for developing adaptive strategies.

Compared to deep learning alternatives, this methodology has comparable performance

but with lower computational cost, making it suitable for real-time applications. Extensive
evaluations using unbalanced FTSE data sets highlight the consistent advantages of the MKL
model for binary and multiclass classification problems. Effectively fuses information from
multiple kernels to address data complexity. Our hybrid approach delivers strong performance
with far less requirement for very large data sets for model training compared to deep learning
methods.

In general, this work provides an efficient and adaptable framework for algorithmic
148 5.6. Summary

trading systems to analyze market microstructure, recognize changing market conditions,

and can be used to dynamically adjust strategies. It provides practical value to traders
by enabling well-informed decisions amid changing regimes. Limitations include model
complexity and interpretability challenges. However, significant contributions to algorithmic
trading underscore the importance of hybrid techniques in understanding market dynamics.
Ongoing research can focus on improving model transparency, evaluating additional
data types, and incorporating alternative kernel learning approaches. In general, this work
provides an essential step toward responsive and insightful modeling of complex financial
markets.
Chapter 6

Statistical investigation in memory and

persistence in interest rate futures con-
tracts

This chapter performs a statistical investigation of long memory and persistence in

orderbook measures at the mesoscale. It reviews relevant literature on multifractal
analysis, information clocks, heavy tails, and kernel two-sample testing. Section 6.4 de-
scribed the high-frequency interest rate futures data used in the study. The experimental
methodology and design are outlined in Section 6.5, focusing on the application of kernel
two-sample testing and the examination of order flow dynamics. Sections 6.6 present
the empirical analysis and results, investigating the multifractal properties, power-law
distributions, and cross-correlations of various microstructure variables across different
information clocks and trading regimes.

149
150 6.1. Introduction

6.1 Introduction

Understanding the long memory and multifractality of microstructure variables in financial

markets is crucial for developing robust trading algorithms and market models. This study
analyses long memory related statistics and information flow dynamics of interest rate futures
at the mesoscale, examining statistical regularities commonly known as stylized facts [8, 162].
Unlike previous studies, we employ information arrival-based sampling instead of regular
calendar intervals [15, 230]. By partitioning each trading day based on variations in the rate
of information flow, we introduce an (event-driven) information clock linked to the mesoscale
concept, and we aim to improve our understanding of the impact of information arrival on
market dynamics. In our case, we consider as information events changes in accumulated
volume traded, market value of volume traded or accumulated number of trades.

Our methodology incorporates multifractal detrended fluctuation analysis (MFDFA) and

kernel two-sample testing (KTST). MFDFA allows us to characterize multifractality and long
memory and analyze leptokurtic behaviour in microstructure variables [112, 113]. The re-
sulting MFDFA-derived variables reveal variations in fractal structure over time intervals
characterized by large and small oscillations [231]. We study power law exponents that
quantify extreme event dynamics for order flow imbalance data alongside MFDFA variables,
such as extended Hurst and mass exponents. Additionally, KTST provides a nonparametric
framework to detect differences between datasets without specific distributional assump-
tions [232, 233].

6.2 Background and hypothesis

We leverage multifractal analysis and kernel two sample testing techniques and information
clocks to examine long memory and scaling behaviours. Key advancements in these areas
are reviewed.
6. Statistical investigation in memory and persistence in interest rate futures
contracts
151

Multifractal detrended fluctuation analysis MFDFA, introduced by Kantelhardt et al. [29,

111], analyzes scaling deviations and removes trends in time series. Jiang et al. [114],
emphasized multifractality in various markets and the integration of MFDFA with complex
financial networks. In addition, they showcase the effectiveness of multifractal analysis in
measuring market inefficiencies and risk, offering a theoretical foundation for comparing
multifractal properties across financial datasets using statistical comparisons. This framework
facilitates the application of kernel-based methods on MFDFA variables.

Financial markets applications Ge [234], identified multifractal cross-correlations in Chi-

nese and US stock market indices, particularly during market crashes, which aligns with our
exploration of how information arrival impacts market dynamics. Similarly, [235] used MFDFA
on interbank lending rates in China, the US, and the UK, finding multifractality that varied
over time and across countries. Also, Zunino et al. [236] found multifractal behaviour in six
Latin American stock market indices, indicating varying degrees of long-range correlations, a
concept we extend to interest rate futures contracts. In Wang et al. [194] the authors examine
long-term cross-correlations and cross-multifractal properties between complex time series.
They apply their methodology to volatility series of stock indices, providing insights into their
multifractal characteristics and cross-correlations.
While the implementation of Thompson et al. [237], examined the equity markets using
segment lengths s from 1 day to 20 days, we adapt the approach to interest rate futures by
considering event-driven sampling. This allows us to capture both intraday and inter-week
scaling behaviours, revealing the multifractal structure of fixed income markets across multiple
time horizons and complementing existing analyses of other assets.
This supports our investigation of the temporal evolution of multifractal properties in order
flow and price dynamics. Aloui et al. [238], studied sectorial indices in the Tunisian stock
market, linking multifractal behaviour to market efficiency and liquidity, paralleling our analysis
of liquidity demand fluctuations and volatility patterns. He et al. [239], examined multiscale
cross-correlations between stock markets in the United States, China, and Europe, noting
stronger correlations at longer time scales and during downturns, reinforcing our findings
152 6.2. Background and hypothesis

on the scaling dynamics and persistence characteristics in different financial instruments.

Furthermore, Begusic et al. [115] provided evidence of power-law behaviour in Bitcoin price
fluctuations, demonstrating heavier tails compared to traditional markets, which informs our
understanding of extreme event dynamics in order flow imbalances. We expand on the
understanding of multifractal structures and long-memory properties in orderbook derived
data, particularly through the lens of information clocks and kernel two-sample testing.

Information clocks Easley et al. [47], modeled information diffusion as a stochastic process
with an underlying information clock. Velasco and Fuentes et al. [1] and Geman [240] showed
stochastic clocks normalizing returns and expressing prices as martingales. Models by
Bouchaud et al. [128] and Mattia [129] linked exponents to order flow and liquidity dynamics.
Hardiman [52] modeled, information arrival as a Hawkes process, finding two linked clock
regimes.

Heavy tails and power laws A power law is a functional relationship between two quantities,
denoted as x and y , such that a relative change in one quantity results in a proportional
relative change in the other quantity, irrespective of their initial magnitudes. This relationship
is mathematically expressed as,
y = axk , (6.1)

where a is a constant of proportionality, x is the independent variable, y is the dependent

variable and k is the power law exponent, determining the nature of the scaling relationship
between x and y . The defining characteristic of a power law is the presence of a nonlinear
scaling relationship between the two variables, governed by the exponent k . When plotted
on a logarithmic scale, the power law function appears as a linear function with a slope equal
to the exponent k . We investigate the presence of power-law distributions in the context of
order flow imbalance in subsection 6.6.2. The analysis of power-law exponents provides
insights into the heavy-tailed nature of financial time series, which is crucial for understanding
extreme events and market inefficiencies.
Similarly, [168] detected power-law tails and asymmetry in the order imbalance distri-
6. Statistical investigation in memory and persistence in interest rate futures
contracts
153

butions of Chinese stocks across different stocks and timescales. Our research extends
these findings by applying power-law fitting to the order imbalance data of futures contracts,
examining how these properties change across different information clocks. The study of
tail exponents has received considerable interest, with methods such as Hurst exponent
calculations; see [96, 241] and Hill estimators [242, 243] being used to validate power laws
over threshold ranges.

Previous works, including those by [244–246], who suggest that power laws persist in
the spectrum of financial time series. In the context of order flow dynamics, a power-law
relationship implies that trading activity, such as the number of trades, is proportional to a
power of the order volume or flow. This relationship has been observed in various financial
markets and is believed to capture the underlying dynamics of order flow [168, 247, 248].

Our study builds on this foundation by analyzing how power-law distributions in order
flow contribute to understanding market microstructure and multifractality (in interest rate
futures contracts). The works of [249] and [168] found heavy-tailed distributions and long
memory in various contexts, including molecule motions and Chinese stock order imbalances.
Hurst exponent estimation and Hill estimators validate power laws [96, 241–243]. Power
laws in order flow dynamics capture trading activity’s relationship with order volume and
flow [247, 248].

Kernel two-sample testing The kernel-based two-sample test (KTST) by [28, 232] is
useful for high-dimensional data analysis, mapping data into a high-dimensional feature
space to assess distributional similarities. Applications include bioinformatics [105], time
series classification [106], and fault diagnosis [107].

However, there are not many known applications for financial times series. Our approach,
which employs kernel two-sample testing for comparing the dynamics of different assets is
analogous to the work of [108], who utilized similar techniques to examine jump components
in jump diffusion models. We utilise KTST to validate models against data while preserving
interpretability [102, 232, 250].
154 6.3. Modelling methodology

Linking stylized facts and MFDFA Stylized facts refer to persistent macro-level phenom-
ena beyond micro-economic theories [8, 126]. Cont [8] examines features like heavy tails
and volatility clustering, while [251] and [252] propose models replicating these properties.
Specifically, [251] proposes an autoregressive model of realized volatility (HAR-RV) to model
the long-memory-like behaviour of volatility. Axioglou and Skouras, [253] find that the long
memory in trade direction is an artifact of daily structural breaks. [254] use MFDFA and
MFDCCA to assess market efficiency across structural breaks, finding that the long memory
in price series is not affected by structural breaks. Schadner et al. [255], used MF-DFA
to study the multifractal properties of market sentiment, finding that market sentiment is
anti-persistent. We adopt definitions from Cont [8], to analyze empirical facts through MFDFA,
linking findings to market dynamics.

6.3 Modelling methodology

We follow a 3-stage approach:

1. We process FIX messages (the financial information exchange (FIX) protocol is a

series of messaging specifications used for price and trade data), and construct
TAQ data (trade and quotes), next, we formulate various statistics that describe the
market microstructure (e.g., price dynamics, volatility, volume, liquidity, order flow). The
microstructure variables, including the definitions of information clock and mesoscale,
are defined in the Supplemental material(see Section S.2). These are then sampled at
different information clocks, establishing robust data capturing mesoscale dynamics
(see Sections 2.1.1 in the Supplement for definitions). In Figure 6.1, this is represented
in steps A, B, C and D.

2. In the second stage, we derive times series of MFDFA variables from the time series of
statistics derived on stage One. Additionally, we fit power-law distributions on order-flow
imbalance data. This is step E in Figure 6.1.

3. Finally, we conduct two-sample nonparametric kernel hypothesis testing on these

6. Statistical investigation in memory and persistence in interest rate futures
contracts
155

fx

| {z } | {z } | {z } | {z }
| {z } | {z }
A. B. C. D. E. F.
FIX TAQ Constructing Information MFDFA KTST
messages Data microstructure clock
variables sampling

Figure 6.1: Starting with exchange message files (A), we assemble a trades and quotes dataset
(B) featuring 14 futures contracts as detailed in Chapter 4. The following step is the
generation of microstructure variables (C), the specifics of which can be found in Chapter 3.
Subsequently, we sample the variables using different information clocks (D) and apply the
MFDFA method (E). The procedure finishes with the implementation of kernel-two-sample
testing (F), the outcomes of which are elaborated in Section C.6.1 of the supplement.

variables to validate market behaviour theories, revealing subtle effects without distribu-
tional assumptions. Additionally, we perform power-law fitting on order book imbalance
data. This is represented in step F, in figure 6.1.

Core methodological ideas

Our methodology focuses on several key aspects:

Statistical validation Integrating MFDFA and KTST offers an approach for extracting vari-
ables to analyze multifractal patterns and persistent correlations and examine distributional
variances in the data sets. MFDFA characterizes multifractal patterns and long-range corre-
lations, while kernel two-sample testing assesses distributional differences within MFDFA
variables across different conditions, enabling hypothesis testing on changes in scaling
behaviour, [256].

Preserving interpretability We prioritize methods that ensure a clear understanding and

interpretation of results, avoiding black-box techniques. The proposed methods maintain
connections to underlying multifractal concepts, facilitating comparisons across information
clocks, activity regimes, or instruments.
156 6.3. Modelling methodology

Handling high dimensionality Kernel testing is suitable for comparing complex empirical
distributions of multifractal variables, handling multifractal spectra and cross-correlations
where parametric methods may struggle [257].

Nonparametric flexibility The data-driven kernel approach offers greater flexibility com-
pared to restrictive distributional assumptions, accommodating the non-linearity and non-
stationarity of the data for more accurate analysis of multifractal behaviours.

Sensitivity Combining MFDFA and kernel testing provides considerable statistical power
to identify subtle differences in multifractal scaling behaviours, long-term correlations, and
memory effects [258].

Regime identification These techniques precisely map scaling dynamics to information

arrival cycles, allowing rigorous classification of different time series regimes [109].

Understanding choices of futures In examining futures, we use a variety of futures

to represent different data elements, emphasizing distinct differences such as long- and
short-memory effects. The subset of futures chosen for this analysis represents the broader
interest rate futures market.

6.3.1 Interpreting the MFDFA statistics

Next, we provide an interpretation of the MFDFA measures (i.e, the output of the analy-
sis) which offers a comprehensive framework for analyzing the complex, scale-dependent
behaviour of the market microstructure variables. Notation follows closely [237] and [29, 111].
They can uncover features like long-range dependence, intermittency, and varying de-
grees of market efficiency that traditional time series analysis methods fail to capture.

Generalized Hurst exponent The generalized Hurst exponent, h(q), is defined as

Fq (s) ∼ sh(q) , (6.2)

6. Statistical investigation in memory and persistence in interest rate futures
contracts
157

where Fq (s) is the q -th order fluctuation function and s is the scale.

log Fq (s)
h(q) = lim . (6.3)
s→∞ log s

Importance: The generalized Hurst exponent h(q), characterizes the scaling behaviour of
financial time series across different moments. It captures both small and large fluctuations,
providing insights into the multifractal nature of market microstructure. For q > 0, h(q)
describes the scaling of large fluctuations, while for q < 0, it describes small fluctuations.
This approach allows us to analyse market dynamics across different scales and magnitudes
of price movements.

Multifractal spectrum

The multifractal spectrum, f (α), is defined via the Legendre transform of τ (q),

f (α) = qα − τ (q), (6.4)

dτ (q)
where α = dq is the singularity strength or Hölder exponent, and τ (q) = qh(q) − 1 is the
scaling function.
Importance: The multifractal spectrum quantifies the distribution of scaling exponents
present in the time series. Its width indicates the degree of multifractality within the mi-
crostructure variables. A broader spectrum suggests a more intricate structure of the un-
derlying process, potentially signaling a more diverse range of trading behaviours or market
inefficiencies.

(Local) Hölder exponent or singularity spectrum

The local Hölder exponent (or singularity spectrum), α(i), at a point i is defined as the
supremum of all h such that,

|X(i + δ) − Pm (t, δ)| ≤ C|δ|h , (6.5)

158 6.3. Modelling methodology

for some constant C > 0 and all δ in a neighborhood of 0, where Pm (i, δ) is a polynomial of
degree m.
Importance: The local Hölder exponent provides a measure of local regularity in the time
series. The multifractal spectrum can provide insight into the microstructure of the market by
uncovering localized scaling properties. This can assist in identifying intervals characterized
by differing degrees of market efficiency, liquidity, or volatility clustering. Over time, fluctuations
in the exponent may signify changes in market conditions or trading dynamics.

6.3.2 KTST Inference procedure

Time series data present challenges for statistical inference due to temporal dependence
and non-stationarity [259]. We employ two-sample testing using kernel methods to compare
′ ′
distributions of MFDFA-generated data samples {x1 , . . . , xn } and {x1 , . . . , xn } from the
probability spaces (S, F, P ) and (S, F, Q).

Two-sample testing Our objective is to determine if the distributions P and Q are equivalent.
We test the hypotheses:

H0 : P = Q, (6.6)

HA : P ̸= Q. (6.7)

We compute a test statistic T and compare it to a threshold, typically the 95% quantile of the
null distribution t0.95 . If T < t0.95 , we reject H0 .

Maximum mean discrepancy (MMD) We use kernel methods to map time series data
to a reproducing kernel Hilbert space (RKHS) and measure similarity. The kernel selection
technique for our two-sample MMD-based tests is based on concepts from [14, 103] and [260].
The MMD test statistic is defined as,

MMD(F, P, Q) = |0µP − µQ | 0F , (6.8)

6. Statistical investigation in memory and persistence in interest rate futures
contracts
159

where µP and µQ are mean embeddings of P and Q in RKHS F . This method maximizes
the (normalized) MMD statistic in order to minimize the type II test error. Effectively, the test
power criterion.

The MMD test is particularly useful for analyzing complex high-dimensional data where
traditional parametric approaches are not applicable. Using a kernel function, the method
captures intricate relationships between data points, enhancing the detection of subtle
differences in distributional properties.

This nonparametric flexibility allows for a more accurate analysis and interpretation of
multifractal behaviours in time series data. In our methodology, we use quadratic MMD test
statistic which is defined as:

MMD2 (P, Q) = Ex,x′ ∼P [k(x, x′ )] + Ey,y′ ∼Q [k(y, y ′ )] − 2Ex∼P,y∼Q [k(x, y)], (6.9)

where k(x, x′ ) is the kernel function.

Procedure for kernel two-sample testing (KTST)

The KTST procedure involved optimizing parameters and testing hypotheses using the
maximum mean discrepancy (MMD) metric. Specific steps, including kernel choice and
hypothesis testing, are detailed in the methodology section. The significance level α of the
tests was set at 5% and 1%, and we computed the p-value, the test statistic, and whether H0
was rejected or not.

This approach ensured reliable detection of differences in multifractal properties across

information clocks. The output of the feature extraction step was subjected to a kernel
two-sample test (KTST). The data were split into training sets (30%) and test sets (70%)
according to a specified ratio. Further details on the KTST methodology are provided in the
methodology section.

Testing procedure Consider a stochastic process X(t), e.g. the microprice or volatility
or order flow. This stochastic process produces the values of the MFDFA variables that are
160 6.4. Data

observed as the information clock changes. Our aim is to use this data in the KTST. The

Algorithm 7 KTST optimizing parameters and testing.

Input: Two samples X, Y , significance level α, and number of test locations J .
1: Split D := (X, Y ) into disjoint training and test sets Dtr and Dte with the same size nte .
2: Optimize parameters θ = arg maxθ λ̂n/2,tr (θ) where λ̂n/2,tr (θ) is computed with the
training set Dtr .
3: Set Tα to the (1 − α)-quantile of χ2 (J0 ).
4: Compute the test statistics λ̂n/2,te (θ) using Dte .
5: Reject H0 if λ̂n/2,te (θ) > Tα .

significance level α of the tests is set at 5% and at 1%, and we compute the p-value, the test
statistic, and whether H0 was rejected or not. As described, we compute the test statistic for
multiple permutations of the data, creating a list of permuted test statistics for evaluation. The
p-value is computed by measuring the proportion of permuted test statistics that are greater
than the observed test statistic. The specific steps are:

• First we define a distance metric on the empirical probability distributions of the various
quantities, based on the distance of their (Hilbert space) mean embeddings. This
metric as described earlier is maximum mean discrepancy (MMD).

• We computed the empirical estimates of MMD across many samples.

• Hypothesis testing is performed by utilising as the test statistic the MMD estimates.
This will determine whether the two samples originate from the same distribution.

Kernel parameter selection The Gaussian kernel with bandwidth determined by the
median heuristic is used. We employ a grid search to maximize the MMD statistic, ensuring
the chosen kernel minimizes the Type II error.

6.4 Data

This section provides details of the irregularly spaced data used throughout this research.
We analyze quote and transaction data, initially collecting raw data and then defining and
6. Statistical investigation in memory and persistence in interest rate futures
contracts
161

selecting a set of events. To ensure universality in our findings, a large number of datasets
and instruments are required. We sample the market at both regular intervals (to study the
effects of changing the information clock) and irregular intervals (as trades have such arrival
rates, and to study effects under the information clock). The dataset comprises top-of-the-
order-book and trades data recorded during 2018. Exchanges typically publish order book
snapshots at intervals of 5 to 200 ms, depending on market activity.
Creating uniformity and comparability within a large dataset poses a challenge. We define
the trade price for an information clock period as the information-event-weighted average
price computed over all trades within that window. If no trade occurs during a window, the
previous valid price observation is used. Trade sizes have different denominations across
markets.

Symbol Asset Type Liquid Market Hours Exchange Number of Days

KE1 3-Year Korea Treasury Bond Future 18:30–17:30 Korea Exchange1 Sunday–Friday
YM1 3-Year Australian Bond Future 24H ASX2 Sunday–Friday
XM1 10-Year Australian Bond Future 24H ASX2 Sunday–Friday
TY1 10-Year T-Note Future 18:30–17:30 CBOT3 Sunday–Friday
TU1 2-Year T-Note Future 18:30–17:30 CBOT3 Sunday–Friday
US1 30-Year T-Bond Future 18:30–17:30 CBOT3 Sunday–Friday
FB1 Long-Term Swiss Government Bond Futures 18:30–17:30 Eurex Sunday–Friday
DU1 Euro-Schatz 2-year Future 1:10–22:00 Eurex Monday–Friday
OE1 Euro-Bobl 5-year Future 1:10–22:00 Eurex Monday–Friday
RX1 Euro Bund 10-Year Future 1:10–22:00 Eurex Monday–Friday
JB1 Japanese 10-Year Bond Futures 8:45–11:02, 12:30–15:02,15:30–5:30 Osaka Exchange Monday-Friday

Table 6.1: Global bonds and futures markets - Regional variations in trading hours, liquidity, and
exchange locations.

Data processing The raw data undergoes several preprocessing steps to ensure accuracy
and consistency:

• Data cleaning: Removal of erroneous records, outliers, and duplicated entries. Trade
and quote records are synchronized to ensure consistent timestamps.

• Time normalization: Data is sampled using various information clocks, such as

event-time, volume-time, and tick-time, to capture different market dynamics.
1
Chicago Board of Trade
2
ASX Trade24 in Sydney, Australia
162 6.5. Experimental setup

• Aggregation: Trades and quotes are aggregated into defined intervals based on the
information clock sampling rate.

Descriptive statistics We present descriptive statistics for the processed data, highlighting
key features such as:

• Trading volume: Average and median trading volumes per interval.

• Price changes: Distribution of price changes, including mean, variance, skewness,

and kurtosis.

• Order flow: Summary statistics of order flow imbalance, including the proportion of
buy and sell orders.

Economic significance of interest rate futures

Although our study examines the full set of futures, we will specifically discuss three key
interest rate futures contracts: XM1, G1, and TY1. These contracts have significant trading
volumes and serve as economic indicators. Hong et al. [261], suggest that open interest in
these futures markets can provide greater insights into future economic activity and asset
prices compared to the futures prices themselves. XM1, G1, and TY1 serve as benchmarks
for their respective markets, influencing various economic factors. In the same work, Hong et
al. [261], also find that open interest in such contracts is procyclical and is correlated with
macroeconomic activity, predicting commodity and bond returns, and short rate movements.
These futures contracts are integral to the global financial system, facilitating risk man-
agement and providing insights into future interest rate movements. Their importance in
shaping economic expectations and financial stability is well established [261–263].

6.5 Experimental setup

We substantiate our contributions through a systematic series of experiments that investigate

the long memory properties and cross-correlations present in the futures data, providing
6. Statistical investigation in memory and persistence in interest rate futures
contracts
163

(a) G1 (b) JB1

(c) FV1

Figure 6.2: The heatmaps provide a visual representation of the correlations between different
microstructure variables across various financial instruments. A positive correlation (red)
indicates that the variables move in the same direction, while a negative correlation
(blue) suggests an inverse relationship. Correlations close to zero indicate no linear
relationship. JB1 shows a notable negative correlation between bid-ask spread and
avgVpT, suggesting that as the spread narrows, the size of the trade increases. G1
exhibits strong positive correlations within its trade-related variables, implying a consistent
trading pattern. FV1’s heatmap suggests a strong coupling between arrival rate and other
variables, particularly with order flow imbalance and trading volume, highlighting closely
tied market participation rates to these metrics.
164 6.5. Experimental setup

empirical validation and informing the proposed theoretical conclusions. Calculations oc-
curred at a lag that shifted forward in information time chronologically across days, with
two-sample tests run sequentially. This allows for the examination of potential modifications
in the microstructure of the market and the multifractal properties intraday and throughout
multiple trading days. Figure 6.3 illustrates the experimental flow. The diagram illustrates
the methodology applied in the experimental process, where trading data is resampled into
discrete information time intervals for each trading day. Each column represents an individual
trading day, and the parentheses indicate the initial and final events within each day. In this
approach, each trading day is resampled based on an information clock, which segments
the day into discrete intervals defined by the flow of market-relevant information, rather than
conventional calendar time.

The columns in the diagram represent individual trading days, with parentheses indicating
the first and last events within each day. The comparison is initially conducted between
consecutive trading days, allowing for an examination of short-term dynamics. As the experi-
mental process progresses, the time gap between the sampled days increases systematically,
facilitating the analysis of longer-term changes in market behaviour.

The sequential nature of the calculations, which shift forward in information time, ensures
that the two-sample tests are performed in a continuous and non- overlapping manner. This
forward-shifting lag in information time enhances the ability to capture evolving patterns and
potential changes in multifractal properties over successive trading days.
6. Statistical investigation in memory and persistence in interest rate futures
contracts
165

Figure 6.3: The diagram shows data resampled into information time intervals across trading days.
Columns represent days, with parentheses indicating information clock events. Two-
sample tests are run sequentially, with calculations shifting forward in information time.
Alternating colors depict the experiment’s progression over successive trading days
resampled into information time.

This experimental setup, as shown in the diagram, allows the investigation of multifractal
properties for various information clocks and trading periods.

6.5.1 Experimental design

This subsection outlines the experimental design used to investigate the microstructure
properties and multifractal characteristics of futures contracts across different information
clocks and trading days.

Data preparation

We assembled a TAQ dataset from exchange message files, including 14 unique futures
contracts focused on interest rates, as detailed in Section 6.4. The data was processed
to generate microstructure variables as described in the methodology section. Features
were extracted using multifractal detrended fluctuation analysis (MFDFA) and multifractal
detrended fluctuation analysis (MFDCCA). MFDFA was applied to the microprice, while
MFDCCA was applied to both the microprice and volatility. Detailed descriptions of these
techniques are provided in the methodology section.
166 6.6. Results

Information clocks

Calculations were performed using three implementations of a stochastic information clock:

• Tick clock: Sampling every 1 trade.

• Volume clock: Sampling every 100 units of volume.

• Dollar volume clock: Sampling every 1000 USD-equivalent in value of volume.

6.6 Results

This section discusses the experimental results. We apply the proposed framework to study
long-term memory and persistence in market microstructure variables using multifractal
analysis.

6.6.1 Experiments on the mass exponent τ and singularity spectrum of the

microprice

The first experiment focuses on the mass exponent (τ ) and the singularity spectrum (α),
derived from MFDFA on micro-prices for different contracts. The decreasing median values of
the kernel matrix (K) with increasing shifts show that these multifractal variables are sensitive
to changes in the information clock. The declining p-values further strengthen the argument
against the null hypothesis, suggesting that the market dynamics change over time and that
past information becomes less relevant for predicting future behaviour. These findings align
with theories that information rates influence exponent values [128].
6. Statistical investigation in memory and persistence in interest rate futures
contracts
167

Shift µ(K) Median p-value Median H0 rejected Median test stat Med Powers
1 0.19 0.17 0.64 0.02 1.53 0.36
2 0.18 0.16 0.70 0.01 1.54 0.37
3 0.23 0.16 0.60 0.02 1.59 0.38
4 0.22 0.14 0.65 0.02 1.56 0.40
5 0.19 0.17 0.61 0.02 1.66 0.38
6 0.19 0.17 0.58 0.01 1.56 0.35
7 0.19 0.18 0.56 0.02 1.55 0.36
8 0.19 0.24 0.53 0.01 1.58 0.37
9 0.19 0.18 0.60 0.01 1.53 0.37
10 0.24 0.16 0.61 0.01 1.53 0.37

Table 6.2: Mean KTST metrics for τ derived from microprice of XM1, G1, and TY1 futures across
shifts 1-10 and all clocks (tick, volume, dollar). Decreasing median kernel matrix values
and declining p-values with increasing shifts support rejection of H0 , consistent across
sampling methods. The consistent patterns exhibited in these observations are true
regardless of the specific sampling method used. Powers is the column of test maximum
power using the best kernel.

Figure 6.4: Results for TY1, XM1, and G1 futures contracts are visualized. Two-sample tests on
tau (Hurst exponent) are shown under tick, dollar, and volume information clocks. Rows
show median kernel-MMD statistic on permuted aggregate data and p-values. Low p-
values reject the null hypothesis, indicating changes in time series dynamics and differing
multifractal behaviour between segments.
168 6.6. Results

6.6.2 Orderflow imbalance

Order flow imbalances indicate temporary order clustering on one side of the orderbook,
leading to pressure on prices and subsequent price movements as liquidity providers adjust
quotes [100]. Significant order flow autocorrelation suggests that microprice changes will
also demonstrate autocorrelation and momentum over a horizon linked to the decay of
the imbalance. Our findings show that the multifractal spectrum of order book imbalance
is skewed towards the left, suggesting that large deviations exhibit greater multifractality
compared to small deviations. This observation aligns with previous findings by [255, 264],
who reported a negatively skewed multifractal spectrum for market sentiment.

Orderflow imbalance and autocorrelation of microprices The autocorrelation profile

of the microprice changes exhibits differences, some decaying faster than others (see
Figure 6.5). The degree of autocorrelation of order flow influences the optimal execution
strategy [265, 266]. Strategies must adapt their market impact models to accurately reflect
the latent properties of order flow.

DU1 Orderflow imbalance DU1 micro price autocorrelation

Figure 6.5: The graph compares microprice autocorrelation functions of different clocks to exponential
decay. Volume and dollar volume clocks show slow decay, indicating strong long memory,
while tick clock decays faster. A vertical line at lag 100 emphasizes the persistent
autocorrelation in volume and dollar volume clocks.
6. Statistical investigation in memory and persistence in interest rate futures
contracts
169

MFDFA for order flow imbalance

Studying the multifractal properties of order book imbalance reveals insights into market
microstructure and mechanics across time horizons. Applying MFDFA to order flow data,
such as order book imbalance and relative tick volume, quantifies scaling behaviours and
distributional divergences when sampled on information clocks. The results reveal significant
heterogeneity in the multifractal properties of order book imbalance time series across all
contracts. However, the rest of the results are less clear. The distributions of singularity
strength α, show statistically significant differences (p < 0.05), with the MMD test statistics
substantially deviating from zero, indicating that certain aspects of multifractality differ signifi-
cantly. In contrast, the distributions of the mass exponent (τ ) fail to reject (p > 0.05), the null
hypothesis of equality, with test statistics approximating zero, suggesting that other aspects
of multifractality remain relatively consistent. The multifractal spectrum and Hurst exponent
distributions exhibit varying degrees of dissimilarity, reflecting the complex structures in the
dynamics of the microstructure of the market, as revealed by the methodology of [103]. We

α τ Spectrum Hurst
Median perm. M M D2 -0.04 -0.00 -0.03 -0.03
2
σ 0.35 2.71 0.22 0.26
µ(Kxy ) 0.64 0.07 0.80 0.74
Mean 0.10 0.01 0.04 0.18
Var 0.00 0.00 0.00 0.00
µ(Kxx ) 0.80 0.08 0.86 0.90
µ(Kyy ) 0.81 0.08 0.84 0.89
Med 0.35 2.71 0.22 0.26
Powers 3.02 0.34 1.95 3.89
Median p-value 0.00 0.45 0.01 0.00
Median test stat 0.21 -0.00 0.05 0.28
Median H0 rejected 0.89 0.36 0.69 0.89

Table 6.3: Results from hypothesis tests comparing the distributions of multifractal variables α, τ ,
multifractal spectrum, and Hurst exponent between order book imbalance time series
across all futures.
170 6.6. Results

can study relationships that are unaffected by clock time by measuring scaling dynamics
based on external information arrival rates. The observed differences in multifractal properties
across financial instruments suggest the need for specific approaches in trading strategies
and risk modeling, while questioning uniform market efficiency assumptions. This finding
highlights the importance of analyzing financial markets at the instrument level.

Figure 6.6: The multifractal spectrum f (α) of the generalized Hurst exponent α for six of the contracts
in our study. The width of each peak provides insight into the complexity and heterogeneity
of intraday trading dynamics.

Power law for OBI

Analyzing the empirical order flow distributions through power law model fitting yields addi-
tional insights into the market dynamics, as discussed in 6.2. Power-law tails indicate heavy-
6. Statistical investigation in memory and persistence in interest rate futures
contracts
171

tailed distributions where extreme values are expected across a wide range of scales [267].

Experimental findings from fitting a Power law We estimate the probability density
function, the cumulative distribution function (CDF), and the complementary cumulative
distribution function (CCDF) using [268]. Examples are presented in Figure 6.7 and in the
supplementary material. The discovery of power-law tails and long-range correlations in

XM1 KE1

Figure 6.7: Plots of probability density functions (top) and cumulative distribution function (bottom)
(CDF) of order book imbalances (OBI) for interest futures. The empirical data (solid line)
exhibits a heavy tail, indicative of a multifractal process.

α σ xmin αSE Dist. D n ntail logLik

OE1 1.15 0.00 0.02 0.00 PL 0.17 26657.0 26657.0 -31587.40
KE1 1.15 0.01 0.02 0.01 PL 0.17 704.0 704.0 -809.52
DU1 1.14 0.00 0.02 0.00 PL 0.17 7463.0 7463.0 -8813.25
XM1 1.11 0.01 0.01 0.01 PL 0.16 151.0 151.0 -158.81
US1 1.16 0.01 0.03 0.01 PL 0.17 307.0 307.0 -352.86
G1 2.13 0.80 0.27 0.80 PL 0.11 2.0 2.0 -0.65
JB1 1.53 0.37 0.08 0.37 PL 0.07 2.0 2.0 -0.70
YM1 1.64 0.45 0.12 0.45 PL 0.07 2.0 2.0 -0.72
FV1 1.15 0.01 0.02 0.01 PL 0.18 260.0 260.0 -288.19

Table 6.4: The power law analysis results on order book imbalance provide additional evidence that
order flow dynamics differ across assets. We test the fit of power law (PL) and logarithmic
distributions and provide the minimal Kolmogorov-Smirnov distance (and the optimal
minimum threshold xmin ), D, between the data and the fit.
172 6.6. Results

order book imbalances aligns with the hypothesis that market microstructures shape complex
system dynamics. Order submission strategies generate fractal structures observable across
inter-event durations [269, 270].

High-average volume per trade versus low-average volume per trade microprice per-
sistence

Motivated by [26] and [271], we modified their analysis by splitting the data into high average
volume per trade and low average size per trade. We compare the sensitivity of the microprice
between the two to test the hypothesized effect. Our work indicates that there was no
statistically significant difference in the Hurst exponents between the high-volume and low-
volume activity regimes. The long-term dependence and predictability of micro prices
remained similar across both high-volume and low-volume days. This contrasts with the
findings of [253], who reported that the long memory in the trade direction is an artifact of
daily structural breaks.

Figure 6.8: MF-DCCA analysis of TY1 future separated by median trading volume. Left: Cross-
correlation ρ distributions; high-volume data shows weaker correlation than low-volume’s
broader range. Right: Fluctuation function F distributions; high-volume data more
concentrated, low-volume broader, indicating different scaling behaviours across volume
regimes.

6.6.3 Multifractal detrended cross-correlation experiments

We conducted experiments analyzing the cross-correlation between various microstructure

variables in different regimes using multifractal detrended fluctuation analysis (MF-DCCA).
These experiments explore their multifractal characteristics and provide insights into market
behaviour at various time scales.
6. Statistical investigation in memory and persistence in interest rate futures
contracts
173

Future MMD Critical Value α Test Result

US1 9.0 × 10−5 1.4 × 10−3 1% Failed to reject
XM1 2.7 × 10−4 1.3 × 10−3 1% Failed to reject
YM1 4.1 × 10−5 1.4 × 10−3 1% Failed to reject

Table 6.5: Two-sample test for the effect of volume on microprice persistence: comparison of MMD
and critical values for US1, XM1 and YM1 futures. The MMD value is lowest for YM1 and
highest for XM1, while the critical values are fairly similar across all three futures. In all
cases, we failed to reject the hypothesis at a 1% significance level.

Microprice versus Garman-Klass volatility

We analyzed the cross-correlation between the microprice and Garman-Klass volatility. The
MFDCCA algorithm measures the correlation between two variables at different time scales
(i.e., window sizes). The cross-correlation effect of trading arrival rates and median traded
volume can provide insights i nto market behaviour. For example, a positive ρ-coefficient
for small window sizes suggests a short-term positive correlation, while a negative DCCA
coefficient for larger window sizes implies a long-term negative correlation.
The table highlights variations in the Hurst exponent derived from the multifractal de-
trended fluctuation analysis (MF-DCCA) of microprice and Garman-Klass volatility across
different future-bar pairings. This indicates the presence of market volatility. For example,
the Hurst exponent for the FV1 volume bar, 1.10, is considerably higher than that of the FV1
tick bar, at 1.06. These findings suggest that these particular futures exhibit a long-memory
effect, particularly evident in the context of volume clock, and therefore these disparities in
the Hurst exponent may occasionally generate opportunities for excess returns.
174 6.6. Results

Bar Future Median Std dev

FV1 1.06 0.13
Tick US1 1.08 0.12
FB1 1.04 0.17
FV1 1.06 0.14
Dollar US1 1.09 0.13
FB1 0.99 0.55
FV1 1.10 0.13
Volume US1 1.07 0.13
FB1 1.05 0.20

Table 6.6: Comparing and contrasting descriptive statistics for the Hurst index of (cross-correlation)
of micro price and Garman-Klass volatility.

Information asymmetry and order flow dynamics

Recent research on market microstructure has provided valuable insights into the information
content of order flow. As hypothesized by [272], order flow imbalance conveys information
about future price movements. Furthermore, trading volume can serve as a proxy for the
informativeness of trades [272]. Building on these findings, [273] suggests that the volatility of
order flow may also capture useful signals about information asymmetry in markets. Therefore,
order flow volatility provides an observable proxy for quantifying the costs of information
asymmetry without directly measuring adverse selection. We hypothesize that lower levels of
mean relative tick volume will be associated with higher OBI, reflecting greater information
asymmetry. This relationship is tested in futures contracts to understand consistency. The
expectation is that the lower average tick volumes will appear to be correlated with the larger
book imbalances [274].

The table of results presents the analysis of order book imbalance (OBI) and mean
relative tick volume correlations. The table’s high Kxy , values (mostly >0.90) suggest strong
relationships between these metrics across futures. The varying test results (0.09-0.37)
indicate different levels of statistical similarity, potentially reflecting market inefficiencies or
trading opportunities. These can reveal lead-lag relationships, with positive values suggesting
bullish trends and negative values indicating market saturation. How large and persistent
6. Statistical investigation in memory and persistence in interest rate futures
contracts
175

Future Mean Mean of Kxy Mean of Kxx Mean of Kyy Mean Gram Med Test Result
DU1 0.00 0.96 0.99 0.94 0.00 0.01 0.13
FB1 0.01 0.90 0.84 0.98 0.01 0.03 0.37
FV1 0.00 0.96 0.97 0.96 0.00 0.01 0.10
JB1 0.00 0.94 0.90 0.98 0.00 0.02 0.16
KE1 0.00 0.96 0.95 0.98 0.00 0.01 0.31
OE1 0.00 0.96 0.99 0.94 0.00 0.00 0.09
RX1 0.01 0.92 1.00 0.86 0.01 0.01 0.31
US1 0.00 0.96 0.97 0.94 0.00 0.01 0.12
YM1 0.00 0.94 0.93 0.97 0.00 0.02 0.13

Table 6.7: MMD-test results comparing median tick volumes and order flows across various futures
contracts. We present mean values for key statistical measures, including cross- and
auto-correlations (Kxy , Kxx , Kyy ), Gram matrices, and test outcomes. These metrics
reveal high correlations among most futures markets. Test results range from 0.09 (OE1)
to 0.37 (FB1), indicating varying statistical similarities across markets.

these correlations are, can determine how traders respond to perceived opportunities.

6.6.4 Discussion

The empirical findings of our work complement and challenge the existing literature. The
presence of power-law tails in order book imbalances aligns with the observations of heavy
tails and long memory in limit order book volume profiles by [275], reinforcing the notion
of markets as complex systems with underlying fractal structures. However, the varying
power-law exponents between assets suggest that the specific mechanisms generating these
heavy tails might be asset-specific, potentially due to strategic trade splitting as hypothesized
by [276].

The results on the multifractal properties of order book imbalance, particularly the dif-
ferences in the singularity spectrum (α) across assets, contribute to the understanding of
market microstructure and order flow dynamics. This finding supports the idea that the
complexity and heterogeneity of intraday trading dynamics vary across different futures
contracts, as suggested by the varying multifractal spectra observed in different markets
and periods by [114]. The analysis of cross-correlations between microprice returns and
realized volatility using MFDCCA reveals a complex relationship between these variables,
176 6.7. Summary

with varying correlations across different time scales and information clock regimes. This
finding extends the work of Corsi [277], on volatility persistence by highlighting the multi-scale
nature of volatility dynamics and the importance of considering different sampling frequencies
when analyzing realized volatility.

Although we did not find statistical differences in Hurst exponents between high- and
low-volume periods, this result could be attributed to the specific characteristics of interest
rate futures markets, which might exhibit different dynamics compared to t he equity markets
studied by [253], where long memory in the direction of trade was found to be an artifact of
daily structural breaks. This discrepancy highlights the need for further research to explore
the impact of trading volume on the long-term dependence of microprices in different asset
classes.

6.7 Summary

We presented research on the microstructure dynamics of interest rate futures using multi-
fractal detrended fluctuation analysis and kernel two-sample testing. Our findings underscore
the presence of long-memory effects and scaling dynamics within microstructure variables,
highlighting variations across different timescale and information arrival rates. We make
contributions to the literature on MFDFA, and confirm findings from [278–281] and others.

The research offers insights into the dynamics and efficiency of financial markets. The
heterogeneity observed in the multifractal properties across futures contracts challenges
the traditional notion of uniform market efficiency, suggesting more complex, asset-specific
dynamics at play. Furthermore, long-range correlations and power-law tails in order book
imbalances support the view of financial markets as complex adaptive systems. The MFD-
FA/KTST framework employed in this work uncovers hidden information about economic
activity and market structures, presenting potential applications in forecasting, risk manage-
ment, and policy formulation. However, future studies are needed to examine multifractal
properties across a broader range of assets and during periods of market stress. This will
improve the predictive capabilities of models and risk assessment, pointing to potential areas
6. Statistical investigation in memory and persistence in interest rate futures
contracts
177

for further research.

We provide empirical evidence that supports the idea that financial markets are complex
adaptive systems with asset-specific dynamics, thereby challenging the traditional assump-
tions of uniform efficiency.
Chapter 7

Conclusions

This chapter concludes the thesis by summarizing key points and challenges in applying
kernel-learning on high-frequency financial time series data. It emphasizes the impor-
tance of information-based sampling methods and presents case studies demonstrating
the effectiveness of the proposed approaches.

We review both academic and industry contributions, alongside practical applications,

while emphasizing how the research advances the theoretical understanding and identi-
fying future directions in the domain.

179
180 7.1. Summary of key ideas

7.1 Summary of key ideas

Kernels play a fundamental role in the machine learning field due to their ability to learn
complicated nonlinear patterns. The main motivating theme of this thesis has been the
application of kernel methods and statistical techniques to the features originating from the
TAQ data, particularly in the first study on generative embeddings derived from microstructure
variables, and in the second, the long-memory properties of MFDFA quantities.
The four core chapters systematically explore the microstructure of financial markets with
a focus on applications for algorithmic trading. Two experimental studies (Chapters 5 & 6)
build on the two previous methdological ones (Chapters 3 & 4). The goal is to provide an
empirical understanding of key market dynamics, founded in extending ideas from the current
literature (Chapter 2) and the methodologies from computational statistics and machine
learning.
This final chapter concludes the thesis by summarising the main contributions made,
emphasizes key findings, and discusses promising directions for future work.

Summary of research findings

1. Hybrid learning:

• This first empirical application facilitates regime classification and trade direction
prediction by leveraging generative probabilistic embeddings derived from HMMs,
which are then used as features for discriminative learning models such as
SVM/MKL.

• This approach addresses the limitations of generative models, which often lack
predictive power, by combining them with discriminative models that enhance
classification accuracy and model interpretability.

2. Kernel-two sample testing on MFDFA quantities of order book variables:

• In Chapter 6, we investigated the multifractal properties and long-range depen-

7. Conclusions
181

dencies in microstructure variables of interest rate futures contracts.

• We applied a two-stage methodology: multifractal detrended fluctuation analysis

(MFDFA) followed by kernel two-sample testing (KTST) to high-frequency quote
and transaction data. We uncovered complex dynamics that challenge traditional
ssumptions of market efficiency and homogeneity.

• Our analysis revealed significant long-memory effects in MFDFA statistics that vary
between timescales and information arrival rates. We observed heterogeneous
multifractal properties among different futures contracts, indicating that market
behavior is not uniform.

• The identification of power-law tails in order book imbalance distributions, with

exponents around 1.15, highlights the prevalence of extreme events in these mar-
kets. Additionally, we detected varying correlations between microprice returns
and realized volatility at different time scales.

• The MFDFA/KTST framework demonstrated its effectiveness in extracting latent

information about market structures, offering potential applications in forecasting,
risk management, and policy formulation.

7.2 Contributions

For academic research This thesis makes several contributions to the research literature
through the application of machine learning and computational statistics techniques. The first
framework proposed develops a hybrid learning approach for classifying market regimes and
predicting trade direction.

In the second empirical study, the application of MFDFA and KTST yields insights into
the long-memory and scaling properties of market microstructure variables. These findings
challenge some existing assumptions about market efficiency and provide a more order-flow-
driven vantage point for the dynamics governing markets.

A key aspect of our methodology is the use of information-based sampling methods,

182 7.3. Further work

which link information arrivals to market dynamics. This approach, departing from traditional
calendar-time sampling, is used in both empirical studies. Enhances our understanding of
how information flow affects markets, significantly contributing to the ongoing debate on
market efficiency and long memory.

Potential applications in Industry The methodologies developed from this research have
direct implications for the trading industry, particularly in the areas of algorithmic trading, risk
management, and market surveillance:

• The hybrid HMM-SVM-MKL framework proposed in this thesis offers a fast and easily
implemented toolkit for the classification of intraday trading regimes, which is critical.

• The ability to classify and predict trade directions with higher accuracy allows trading
systems to adapt more effectively to the rapidly changing conditions of financial markets,
thus improving trade execution and profitability.

7.3 Further work

Here, we outline potential directions for future research.

7.3.1 Hybrid deep learning models

Integration of deep kernels: We proposed hybrid deep kernel learning, which adapts
traditional kernel methods by parameterizing them with deep neural networks [282, 283].
This approach combines the flexibility of deep neural networks with the approximation
power of kernel methods. The proposed algorithm could involve multiple kernels and be
implemented and trained end-to-end using an SGD-type algorithm; see [284, 285]. Adapting
techniques from [286], could improve the performance of deep hybrid kernel methods,
enabling realistic implementations.
This direction, also motivated by Azim [83] further explores generative feature separability
to improve kernel classifier accuracy while potentially reducing computational complexity.
7. Conclusions
183

7.3.2 Connecting the multifractal spectrum and information theoretic mea-

sures of market efficiency

This research direction seeks to connect multifractal analysis and information theory. The
multifractal spectrum describes the scaling of the TAQ data, while entropy and mutual
information quantify market predictability and efficiency.
Entropy and fractal dimensions are related complexity metrics. Rényi entropy derives the
generalized correlation dimension for multifractals. MFDFA, which estimates the multifractal
spectrum, is ultimately linked to entropy measures. The local time series properties can
be characterized by this spectrum, which would allow MFDFA to evalute signal complexity
related to entropy [287].
A potential approach would be as follows:

1. Deriving relationships between the width of the multifractal spectrum and measures of
market entropy.

2. How the asymmetry of the multifractal spectrum relates to the directionality of informa-
tion flow in markets.

3. A unified framework that expresses market efficiency in terms of both multifractal

characteristics and information theoretic quantities.

By extending the work of Yujun et al [288]., we would include entropy-based measures

that bridge MFDFA and information theory, revealing insights into market efficiency and
predictability.

7.3.3 Non-stationary extensions of kernel two-sample tests for analyzing

regime changes

The kernel two-sample tests used in the thesis assume stationarity, which may not hold
during regime changes. Extending these tests to non-stationary settings would increase their
applicability for detecting and analyzing market regime shifts. There are a few directions we
can explore:
184 7.3. Further work

• Developing time-varying kernel functions that can adapt to changing data distributions.

• Incorporating change point detection methods into the kernel two-sample testing
framework.

• Exploring online or sequential versions of the test that can handle streaming data.

These extensions would allow for more robust detection of regime changes in real-time
market conditions.
Chapter A

Limit order book

A.1 Limit order book

The LOB is the main trading mechanism for modern market dynamics, consolidating and
organizing all types of orders. Currently, these are mainly electronic trading platforms that
are used by financial markets around the world. The record of all buy-and-sell interests in a
particular instrument is made available to all market participants. Trade requests, which are
called bids (for a seller looking for a buyer) and offers (for a buyer looking for a seller), are
accompanied by a specified quantity q and an upper or lower limit pbid or ask for the price to be
executed. The LOB organizes these orders primarily by price and then by the submission
time of the order [289].

Order types Participants in the market have the option of two different kinds of order in an
LOB: market order and limit order. Limit orders can be defined as the instruction to trade a
specific instrument with a specified quantity q at a predetermined price plimit or better. Any
limit orders that are not immediately executed enter the LOB into a predefined queue and
essentially form the "book". An LOB is therefore a structured record with a schedule of buy
and sell quantities of an instrument organized by the ordering of prices at different levels.
Therefore, all limit buy-sell orders constitute the full order book at various levels. Order books
can be obtained at various levels of detail. For example, Level-1 order book contains the

185
186 A.2. Mathematical description of a limit order book

most advantageous price level of the order book, while Level-2 order book provides the five
to ten highest levels of prices and quantities on the bid and ask side; see [290, 291]. Market
orders, which can be defined as mbid/sell
t , are executed against these resting orders. Market
orders are essentially orders to trade certain amounts of security at the best price available in
LOBs immediately; see [290, 291]. When an order is matched with the best available price,
this leads to immediate trade execution, and the LOB is updated accordingly.

Figure A.1: This image illustrates the condition of a market’s order book in relation to the flow of
orders, both incoming and outgoing, and provides a visual representation of the current
status of orders. IN flows refer to the influx of new orders and modified orders, hence
resulting in an accumulation of resting orders within the order book at different levels.
On the right side, OUT flows refer to orders that have been executed or canceled and
subsequently leave the order book (and the captured trade record).

A.2 Mathematical description of a limit order book

LOB can be represented as a stochastic process.

Definition 6 (Limit Order Book (LOB)). A Limit Order Book (LOB) is a structured list main-
tained by an exchange that captures unexecuted limit orders for a specific financial instrument.
Each order in the LOB is characterized by a price, quantity, and side (either ask or bid).
The LOB is organized into discrete price levels, indexed by l = 1, 2, . . . , n. These levels
are bifurcated into two primary sides: the bid side representing buy orders, and the ask
side representing sell orders. For each side of the LOB and at a specific time ti , orders are
aggregated by their price levels, leading to a volume associated with each level. This volume
is denoted by qlbid (ti ) for bids and qlask (ti ) for asks.
A. Limit order book
187

Figure A.2: Visualizing the depth of the order book: An illustration of a stylized limit-order book is
presented, with ten levels on each side. The green levels are the bids, with the highest
bid closest to the mid price indicated with an arrow, while the lowest ask close to the mid
price is indicated again with an arrow, indicating the top of the book. The red bars indicate
the quantities and levels of the offer. The liquidity on either side increases gradually, with
the cumulative volume indicated by dotted lines.

The price levels are organized on the basis of competitiveness. On the ask side, the
most competitive price is the lowest, representing the best price, denoted as pask
1 (t). On

the contrary, on the bid side, the highest price is the most competitive, representing the
best buying price, and is given by pbid
1 (t). Hence, the following definitions can be derived

explicitly [292]:

• q ask (pask , t) - Volume available on the ask side at price level pask and time t

• q bid (pbid , t) - Volume available on the bid side at price level pbid and time t

• mask
t - Market order volume arriving on the ask side at time t

• mbid
t - Market order volume arriving on the bid side at time t

The progression away from these best levels leads to prices that are considered less com-
petitive but might provide deeper liquidity due to the accumulated volume of orders at those
specific levels.
188 A.2. Mathematical description of a limit order book

top
The spread at the top of the book can be defined as follows: sti = pask bid
1,ti − p1,ti . Based

on Definition 6, the dynamics of the evolution of the order book as events happen:

q ask (pask , t + ∆t) = q ask (pask , t) + δq ask (pask , t, t + ∆t) − ϵq ask (pask , t + ∆t). (A.1)

q bid (pbid , t + ∆t) = q bid (pbid , t) + δq bid (pbid , t, t + ∆t) − ϵq bid (pbid , t + ∆t). (A.2)

where δq(p, t, t + ∆t) is the change in volume at the price level p (in the bid or ask, respec-
tively), due to new limit orders in [t, t + ∆t] and ϵq(p, t + ∆t) is the change due to order
executions against market orders [13, 293]. The bid and ask prices are updated as follows.

pask
t+∆t = min{p : q
ask
(p, t + ∆t) > 0} (A.3)

pbid bid
t+∆t = max{p : q (p, t + ∆t) > 0}. (A.4)

This stylized stochastic model captures the evolution of the LOB, driven by the arrival of limit
and market orders. Key quantities such as spread, depth, and liquidity can also be derived
from this representation. Certain quantities of interest are defined in Section 4.3.

Orders: The building blocks of limit order books

In this section, we further discuss and in more detail the two main types of orders that shape
an LOB. First, we set the notation.

• si or s′i ∈ {A, B} represents the side, with A for ask and B for bid

• q ask (p, t) is the volume on the ask side (sellers) at price p and time t

• q bid (p, t) is the volume on the bid side (buyers) at price p and time t

• qlsi (t) represents the volume at price level l and side si at time t

For example, a limit order LOB update, at level li is:

qlbid
i
(ti ) = qlbid
i
(ti−1 ) + qi
A. Limit order book
189

Figure A.3: Flow chart of the various stages of data processing. To obtain the data set that contains
the most accurate information possible about the physical LOB state in any millisecond
time window, additional data processing steps are needed, which are summarized in this
flow chart.

This means new bid volume qi was added at the level li . The superscripts track the bid and
ask sides independently as the LOB evolves. Now we present the definition of two types of
orders in the order book.

• Limit orders - Orders submitted with a price limit pbid ,

which add volume q ask (p, t) or q bid (p, t) to the LOB.
A limit order can be represented by a 4-tuple (pbid/ask , q, T, ω) where pbid/ask is the limit
price threshold, q is the order size, T is the expiration time, and ω is the outcome.

• Market orders This type of order is to be executed immediately at the best available
price in the LOB. A market order is a 2-tuple (q, ω) where q is the order size and ω is
the outcome.

Limit orders remain in the LOB until they are executed against incoming market orders mbid
t

or canceled (’gtc’, good till cancel). Volume q ask (p, t), q bid (p, t) available at each price level
190 A.2. Mathematical description of a limit order book

comprises overlapping limit orders at that price. Market orders consume volume from the
opposite side of the LOB. For example, a buy market order consumes volume from the ask
side, starting from the lowest ask price pask
t .

A.2.1 Mathematical framework for limit order book dynamics

The LOB transformations upon the arrival of an event at time ti are,

• Limit order: q s l(ti ) = q si li (ti−1 ) + qi

′ ′
• Market order: q1s (ti ) = max q si 1(ti − 1) − qi , 0 where s′i ̸= si

• Cancellation: q si li (ti ) = max q si li (ti−1 ) − qi , 0

Post-operation recalibration of bid/ask prices is necessary to maintain the spread and order
in the LOB. Sides are given by s (limit order), or s′ (market order) and level by l.

Dynamics of LOB updates

After these operations, a necessary recalibration of the bid and ask prices is crucial to uphold
the integrity of the spread and maintain the requisite order in the LOB.

Extending the model towards the state and behaviour of a real-world LOB Below,
some extensions are highlighted to make the above LOB model more realistic.

• Markets have specific external events, like scheduled financial news or

geopolitical events, which can influence order submission and trading patterns.
Including some model of such events would be beneficial, for example, as in [294].

• Understanding price tension: the behaviour of volume distribution at different price

levels is essential to understand
the availability of liquidity in the market [295].

• Strategic order placements, such as high-frequency trading strategies,

or volume-weighted average price (VWAP), can greatly influence order flow.
Chapter B

Hidden Markov models

B.1 Introduction

We present an overview of a class of discrete state latent variable models used in probabilistic
machine learning, called hidden Markov models (HMMs). These models encapsulate the
interactions among variables to synthesize potential states and are pivotal in fields like speech
recognition, bioinformatics, finance, and more. It is also a topic well covered in literature, thus
this section is compact and presents an overview. We guide the reader towards the following
excellent (but not exhaustive) list [147, 296–298].

Generative methods model the joint distribution P(x, y) of input and output variables,
allowing for the generation of samples from this distribution. They describe the probability
density function associated with observations, enabling the random generation of observation
sequences and the calculation of sequence likelihoods.

B.2 Hidden Markov models for sequential data modeling

HMMs are generative statistical models that incorporate discrete latent variables into a
defined probability space, along with Markov chains relating hidden states to associated
observation probabilities.

191
192 B.2. Hidden Markov models for sequential data modeling

Assumptions and simitations HMMs are built on two key assumptions:

• The Markov property: The probability of transitioning to the next state depends only
on the current state and not on the sequence of states that preceded it.

• Conditional independence: Given the current state, the observations are conditionally
independent of previous states and observations.

While these assumptions enable efficient algorithms, they may not fully capture complex
temporal patterns or long-term dependencies in the data. For instance, the conditional
independence assumption may not apply when observations depend on multiple hidden
factors.

B.2.1 Problem definition and notation

Consider an HMM with discrete hidden state space Z = {ζ1 , ζ2 , . . . , ζK } and observation
space O. The HMM is parameterized by λ = (π, A, B), where π is the initial state distribution,
A is the state transition matrix, and B defines the observation likelihoods.

B.2.2 Key HMM inference problems

There are three fundamental inference problems for HMMs:

1. Evaluation: Compute the probability of an observation sequence given the model,

P(O|λ).

2. Decoding: Find the most likely sequence of hidden states given the observations,
Z∗ = argmaxZ P(Z, O|λ).

3. Training: Estimate the model parameters λ that maximize P(O|λ) from training data.
B. Hidden Markov models
193

Evaluation of P(O|λ)

Given an observation sequence O and HMM λ, the forward algorithm computes the observa-
tion likelihood efficiently using dynamic programming:

αt (i) = P(O1 , . . . , Ot , Zt = ζi |λ)

with recursion:

α1 (i) = πi Bi (O1 )
K
X
αt (i) = αt−1 (j)Aji Bi (Ot )
j=1

for t = 2, . . . , T and i = 1, . . . , K . The probability of the observation sequence is:

K
X
P(O|λ) = αT (i)
i=1

The complexity is O(K 2 T ), where K is the number of states.

Finding the optimal state sequence

The Viterbi algorithm finds the optimal hidden state sequence Z∗ by maximizing the joint
probability P(Z, O|λ):

δt (i) = max P(Z1 , . . . , ζt = ζi , O1 , . . . , Ot |λ)

ζ1 ,...,ζt−1

with recursion:

δ1 (i) = πi Bi (O1 )

δt (i) = max [δt−1 (j)Aji ]Bi (Ot )

1≤j≤K
194 B.3. Tractable algorithms for inference, decoding, and learning

for t = 2, . . . , T . The optimal state sequence is recovered via backtracking. The complexity
is O(K 2 T ).

Training the model

The Baum-Welch algorithm uses an iterative Expectation-Maximization (EM)algorithm to

re-estimate the parameters. In the E-step, it computes the auxiliary variables:

γt (i) = P(ζt = ζi |O, λ)

ξt (i, j) = P(Zt = ζi , ζt+1 = ζj |O, λ)

In the M-step, the parameters are re-estimated as:

π̂i = γ1 (i)
PT −1
ξt (i, j)
Âij = PT −1t=1
PK
t=1 k=1 ξt (i, k)
PT
t=1 (Ot = k)γt (i)
B̂i (k) = PT
t=1 γt (i)

This process repeats until convergence.

B.3 Tractable algorithms for inference, decoding, and learning

Hidden Markov models offer flexible frameworks for modelling sequential data. However,
it requires efficient inference, decoding, and parameter learning algorithms. The forward-
backward algorithm enables tractable inference and smoothing in HMMs by exploiting their
conditional independence structure. The Viterbi algorithm provides an efficient dynamic
programming approach for finding the most likely state sequence. The Baum-Welch algorithm
leverages forward-backward computations for parameter estimation to maximize the likelihood
iteratively.

In this section, we provide a basic outline for the classes of algorithms used in these
B. Hidden Markov models
195

topics.

Forward-Backward algorithm

The Forward-Backward algorithm computes the likelihood of the observation sequence

P (O|λ) and the posterior probabilities for the hidden states P (Zt = ζi |O, λ).

A12 A23

ζ1 ζ2 ζ3

O1 O2 O3

Figure B.1: A HMM with states ζi that emit variables Oi , i ∈ {1, 2, 3}. The transition probability
between two states ζi and ζj is Aij .
196 B.3. Tractable algorithms for inference, decoding, and learning

Algorithm 8 Forward-Backward algorithm for HMM state estimation

Input: Sequence of observations O, set of states Z , known parameters λ = {π, A, B}
1: Initialization: Initialize the forward probabilities α(1, i) = π(i)Bi (O1 ) for all states i.
2: for t = 2 to T do
3: for each state i do
4: Compute the forward probabilities:

|Z|
X
α(t, i) = Bi (Ot ) Aji α(t − 1, j)
j=1

5: end for
6: end for
7: Initialization: Initialize the backward probabilities β(T, i) = 1 for all states i.
8: for t = T − 1 downto 1 do
9: for each state i do
10: Compute the backward probabilities:

|Z|
X
β(t, i) = Aij Bj (Ot+1 )β(t + 1, j)
j=1

11: end for

12: end for
13: Compute the likelihood of observations:

|Z|
X
P(O|λ) = α(T, i)
i=1

14: For each state i and time t, compute the posterior probability:

P(Zt = ζi |O, λ) ∝ α(t, i)β(t, i)

15: return Forward probabilities α, backward probabilities β , and posterior probabilities

P(Zt = ζi |O, λ) for all states and times.
B. Hidden Markov models
197

Viterbi algorithm

The Viterbi algorithm finds the most likely sequence of hidden states given an observation
sequence and an HMM. It uses dynamic programming to avoid the computationally prohibitive
task of evaluating all possible state sequences. It is a decoding algorithm.

Algorithm 9 Viterbi algorithm for most likely sequence estimation

Input: Sequence of observations O, set of states Z , start probabilities π , transition probabil-
ities A, and emission probabilities B
1: Initialize the Viterbi matrix δ(0, i) ← π(i) × Bi (O1 ) for all states i ∈ Z .
2: Initialize the backpointer matrix ψ(0, i) ← 0.
3: for t = 2 to T do
4: for each state j ∈ Z do
5: δ(t, j) ← maxi∈Z (δ(t − 1, i) × Aij × Bj (Ot ))
6: ψ(t, j) ← argmaxi∈Z (δ(t − 1, i) × Aij )
7: end for
8: end for
9: P∗ ← maxi∈Z δ(T, i) {Probability of the optimal path}
10: path(T ) ← argmaxi∈Z δ(T, i)
11: for t = T − 1 downto 1 do
12: path(t) ← ψ(t + 1, path(t + 1))
13: end for
14: return Optimal path path

Baum-Welch algorithm

The Baum-Welch algorithm estimates HMM parameters λ by maximizing P(O|λ) using the
expectation-maximization (EM) algorithm. It’s a learning algorithm.
They are often used in conjunction: Baum-Welch to train the model, and then Viterbi to
use the trained model for inference on new data.

Applications HMMs enable modeling and analysis of sequential data across domains
including:

• Speech recognition

• Computational biology
198 B.3. Tractable algorithms for inference, decoding, and learning

Algorithm 10 Baum-Welch algorithm for HMM parameter estimation

Input: Sequence of observations O, set of states Z , initial estimates of start probabilities π ,
transition probabilities A, and emission probabilities B
1: while convergence criterion is not met do
2: E-step:
3: Compute the forward probabilities α for each state and time step using the observations
and current estimates.
4: Compute the backward probabilities β for each state and time step using the observa-
tions and current estimates.
5: Compute the expected number of transitions ξ(i, j) from state i to state j using α, β, A,
and B.
6: Compute the expected number of times γ(i, t) the HMM is in state i at time t using α
and β .
7: M-step:
8: Update start probabilities π(i) ← γ(i, 1).
PT −1
ξ(t,i,j)
9: Update transition probabilities Aij ← Pt=1 T −1 .
t=1
γ(t,i)
P T
(Ot =k)γ(t,i)
10: Update emission probabilities Bi (k) ← t=1 PT .
t=1
γ(t,i)
11: Compute log-likelihood L = log P(O|λ).
12: Check for convergence using change in L or other criterion.
13: end while
14: return Updated parameters λ = {π, A, B}

• Finance and econometrics

• Signal processing

• Anomaly detection

Tasks include likelihood evaluation, state inference, parameter estimation, forecasting, and
more.

B.3.1 Advances in HMM training

Recent advances have aimed to improve HMM training and parameter estimation beyond
traditional Baum-Welch EM:

• Spectral learning algorithms estimate HMM parameters through matrix decomposition

methods, avoiding iterative EM procedures. Spectral learning algorithms offer a com-
B. Hidden Markov models
199

pelling alternative to traditional EM methods, especially when dealing with large-scale

HMM applications where computational efficiency is of importance.

• Kernel embedding techniques implicitly map sequential data into high-dimensional

feature spaces to learn non-linear HMMs.

• Neural HMMs leverage deep neural networks to model emission densities, capturing
complex data distributions.

• Nonparametric Bayesian HMMs place priors directly over transition and emission
distributions, avoiding specifying their forms.

• Structured sparse methods induce sparsity in HMM parameters via regularization for
interpretability.

These innovations can enhance model accuracy, flexibility, scalability, and computational
performance compared to standard HMM training procedures. However, basic EM methods
remain effective and widely used in many simpler applications.

Comparison with related models

Hidden Markov models (HMMs) extend simple Markov chains by incorporating hidden states,
which allow for the modeling of more complex temporal dependencies. Markov chains model
the sequence of states directly observable, while HMMs assume that the observed sequence
is generated by a sequence of hidden states. This distinction makes HMMs more flexible in
modeling scenarios where the underlying states are not directly observable. Unlike mixture
models, which assume that each observation is independent and identically distributed,
HMMs capture sequential dependencies between observations, making them particularly
suitable for time series data.

Extensions and variants Several extensions and variants of the basic HMM have been
developed to address its limitations and to model more complex scenarios:
200 B.3. Tractable algorithms for inference, decoding, and learning

• Hidden semi-Markov models (HSMMs): These models extend HMMs by explicitly

modeling the duration of time spent in each state, allowing for more flexible state
duration distributions [299].

• Factorial HMMs: These models use multiple independent Markov chains to represent
the hidden state, enabling the modeling of more complex dependencies between
observed sequences [300].

• Coupled HMMs: These models allow for the interaction between multiple HMMs,
making them suitable for modeling systems with interacting processes.

• Hierarchical HMMs: These models introduce a hierarchical structure into the hidden
states, enabling the modeling of data with inherent hierarchical dependencies [301].

Computational considerations The computational complexity of standard HMM algo-

rithms such as the Forward-Backward, Viterbi, and Baum-Welch algorithms O(K 2 T ), where
K is the number of states and T is the length of the sequence. This quadratic dependence
on the number of states can be computationally intensive for models with a large number of
states or large datasets. To improve scalability, techniques such as beam search, variational
inference, and spectral learning methods can be used. Spectral learning methods, for in-
stance, rely on singular value decomposition (SVD) and offer computational efficiency while
providing provable guarantees under certain conditions

Considerations for practical application We provide a number of considerations for the

application of HMMs in practice. Also, we highlight the importance of cross-validation and
proper tuning of hyperparameters to achieve robust models.

• Choose the number of hidden states based on domain knowledge or use model
selection criteria such as Bayesian information criterion (BIC) or Akaike information
criterion (AIC).

• Use log-probabilities to prevent numerical underflow in long sequences.

B. Hidden Markov models
201

• Handle missing data by marginalizing over possible values in the Forward-Backward

algorithm.

• Initialize parameters carefully to avoid local optima during training. Techniques such as
multiple random starts and leveraging domain-specific information can be beneficial.

Recent developments Recent advances in HMM research include:

• Spectral learning algorithms for efficient parameter estimation without the need for
iterative methods like EM.

• Kernel embedding techniques for extending HMMs to non-linear settings.

• Neural HMMs that leverage deep learning for modeling emission probabilities, enhanc-
ing the capability to capture complex dependencies in the data.

• Nonparametric Bayesian approaches that allow for more flexible model specifications,
adapting the number of states based on the data.

• Structured sparse methods that improve interpretability and performance by incorporat-

ing sparsity in the transition and emission matrices.

Evaluation metrics We present a brief list of metrics appropriate for evaluation of the
performance of HMMs [296, 302]. These metrics help in assessing the model’s ability to
capture the underlying dynamics of the data and its generalization capability to unseen
sequences:

• Log-likelihood of held-out data: Measures how well the model predicts unseen data.

• Perplexity: Used for sequence modeling tasks, indicating the model’s uncertainty.

• Task-specific metrics: Such as classification accuracy or prediction error, depending on

the application.

• Model selection criteria: Such as BIC or AIC, to choose the best model among a set of
candidates.
Chapter C

Kernels

C.1 Background on kernels

This appendix demonstrates the fundamental principles of kernel learning, with a focus on
the Representer theorem. We establish the theoretical framework of reproducing kernel
Hilbert spaces, which provides the essential context for applying the Representer theorem. A
comprehensive understanding of RKHS properties and nuances is crucial for appreciating
the implications and practical utility of the theorem in kernel-based learning algorithms. Given
that exploring all these topics would be an overly ambitious task, we provide references for
further study.

C.1.1 Reproducing kernel Hilbert spaces

Research on reproducing kernel Hilbert space (RKHS) and related topics has gained pop-
ularity in statistics and machine learning due to the success of kernel-based methods in
solving a variety of problems, from prediction to classification [139]. These methods exploit
the rich representation of non-linear feature maps. We provide a simple definition of RKHS
sufficient for our needs. For a more detailed treatment of RKHS theory, the reader is referred
to [303, 304].
Let H be a Hilbert space of real-valued functions defined on a nonempty set X , associated
with the inner product ⟨·, ·⟩H . A function k : X × X → R is said to be a reproducing kernel of

203
204 C.1. Background on kernels

H if and only if:

Definition 7. Reproducing kernel [303]

1. (Inclusion) ∀x ∈ X , k(x, ·) ∈ H

2. (Reproducing Property) ∀x ∈ X , ∀f ∈ H, ⟨f, k(x, ·)⟩H = f (x).

The reproducing property states that the value of the function f at x is obtained by taking
the inner product of f with the function k(x, ·). This allows for the evaluation of the bivariate
kernel function.
k(x, x̃) = ⟨k(x, ·), k(x̃, ·)⟩H , ∀(x, x̃) ∈ X × X . (C.1)

Throughout this thesis such a bivariate function is symmetric, i.e.

∀(x, x̃) ∈ X × X ,

k(x, x̃) = ⟨k(x, ·), k(x̃, ·)⟩H

= ⟨k(x̃, ·), k(x, ·)⟩H = k(x̃, x).

A Hilbert space equipped with the symmetric reproducing kernel k is called a reproducing
kernel Hilbert space (henceforth RKHS) [304], or a proper Hilbert space [303]. Sometimes
we use Hk to explicitly refer to the RKHS H equipped with kernel k .

Definition 8. Positive-definite functions [303]

A function k : X × X → R is called positive definite if:

X
∀n ≥ 1, ∀a1 , . . . , an ∈ R and ∀x1 , . . . , x ∈ X , ai aj k(xi , xj ) ≥ 0 (C.2)
i,j

Positive definite kernel and RKHS Assume that k : X × X → R is positive definite. Then:
C. Kernels
205

• There exists a map

ϕ : X → H, not unique : ∀x, x̃ ∈ X, , k(x, x̃) = ⟨ϕ(x), ϕ(x̃)⟩H . (C.3)

Theorem 1. Moore-Aronszajn theorem [305]

There is a unique Hilbert space H of functions on X where k is the reproducing kernel.

Following from this theorem, we know that the RKHS, H, induced by a positive definite
reproducing kernel, is unique. Conversely, if a Hilbert space of functions on a nonempty set is
equipped with a reproducing kernel, then such a kernel is also unique [303, 304]. Therefore,
given a characteristic kernel k : T × T → R, we denote by H its corresponding RKHS. The
space H is a Hilbert space with inner product ⟨·, ·⟩H and norm ∥ · ∥H . For any two function
values, their covariance is defined as Cov[f (x), f (x′ )] = k(x, x′ ).

C.1.2 The Representer theorem and its implications for kernel machines

The Representer theorem is fundamental in understanding regularization functionals in po-

tentially infinite-dimensional spaces and developing kernel methodologies. It shows that
many analytical problems can be solved using a kernel expansion dependent on training data
examples. This transforms computationally challenging problems in vast spaces into man-
ageable optimization issues focused on scalar coefficients. While Kimeldorf et al. introduced
the theorem in [306], its widespread application across various domains began in the late
1980s, driven by the availability of complex datasets and computational advancements.

In scenarios where the goal is to determine a function f from a designated model space
using a given dataset, learning within infinite or vast dimensional spaces often presents as
an ill-posed problem. Regularization is frequently employed to address this issue. Formally,
a regularization functional can be defined as a map from the model space to the real line,
delineated as:

Cost(f |data) + λG(f ). (C.4)

206 C.1. Background on kernels

In the above expression, Cost(f |data) quantifies the cost function, signifying the model’s
alignment with the data, while G(f ) acts as a penalty term, in order to prevent overfitting and
integrate prior knowledge such as the desired function’s smoothness. Hyperparameter λ
manages the compromise between these two competing terms. In [306], the authors utilized
the Sobolev space as a model space in their influential study. They used the least squares
method as their cost function, with the penalty term defined as J(f ) = ab (Lf )2 dx, where
R

L denotes a linear differential operator [307]. Numerous subsequent examinations have

thoroughly examined the alterations and enhancements made to these components, with the
aim of solving different tasks [137, 307, 308].

A significant area of research has been dedicated to determining the essential and
complete conditions that must be met for the theorem to be applicable [137, 309].

The Representer theorem provides a way to measure kernel evaluation accuracy and
identify the most suitable kernel for a given problem. It’s closely linked to reproducing
kernel Hilbert spaces (RKHS), which have unique properties that help determine the most
appropriate approach to learning problems involving kernel evaluations at data points.

The theorem utilizes the continuity of point evaluation in RKHS and the kernel function’s
structural characteristics. RKHS properties ensure the optimal function is close to given
data, enabling representation in a specified format. It connects high or infinite-dimensional
feature spaces caused by kernels to computationally feasible machine learning algorithms
like support vector machines.

Within regularized empirical risk minimization, the Representer theorem expresses RKHS
properties and organization. The interconnection between these subjects is fundamental, as
RKHS forms the framework upon which the theorem is based on.

C.1.3 Formulation of the learning problem within the context of RKHS

Consider the supervised learning paradigm with training samples {x1 , x2 , . . . , xn } in input
space X and labels {y1 , y2 , . . . , yn } in label space Y . For simplicity, let X = Rd and Y =
{−1, 1}. The objective is to determine a predictive mapping f : X → Y from a hypothesis
C. Kernels
207

space H ⊂ Y X , minimizing error on a novel data set (xn+1 , yn+1 ), . . . , (xn+l , yn+l ) from the
same distribution. In kernel methods, a common choice for the hypothesis space H is an
RKHS, associated with a positive definite kernel. This choice of RKHS offers a rich and
flexible framework for constructing complex non-linear predictive mappings, while maintaining
desirable mathematical properties such as completeness and the reproducing property.

The Representer theorem ensures that the solution in a RKHS, can be expressed as
a linear combination of kernel functions centered on the training data. This simplifies the
learning problem. For non-trivial learning tasks, an inherent relationship between the training
and novel datasets is assumed, with each data tuple (xi , yi ), independently sampled from a
stationary probability distribution P .

In this framework, the efficacy of the predictive function f can be quantitatively evaluated
by the expected error rate, expressed as:

E(x,y)∼P [I{f (x) ̸= y}] . (C.5)

where I{·} denotes the indicator function.

C.2 Learning, testing and expressing structure with kernels

In Chapter 5, we focus on the problem of combining multiple sources of information to make

a learning decision within the framework of supervised learning. Thus, the specific research
question concerns the classification of input samples utilizing feature sets derived from varying
temporal sources within the scope of financial time series. This section begins with a simple
linear regression example and moves on to the background knowledge for classification,
kernel methods, conclusions, and readers. For a more extensive and detailed background,
the reader is directed to the following references (nonexhaustive list): [139, 140, 310–312]. In
kernel methods, feature mappings implicitly map input patterns to high-dimensional feature
spaces. kernel functions then measure how similar mapped features are to each other.

Appropriate kernel function selection is critical, as poor choices can substantially degrade
208 C.3. Expressing structure through kernels

model performance. Manual search for an optimal kernel is computationally prohibitive and
unlikely to yield the globally optimal solution. Multiple kernel learning (MKL) methods learn a
composite kernel as a linear combination of base kernels to avoid exhaustive kernel tuning.
MKL aims to leverage complementary information from different kernels and automate the
search for an enhanced kernel, overcoming the limitations of manual kernel engineering.
This topic is discussed in detail in the following sections.

C.3 Expressing structure through kernels

This section provides an overview of the basic kernel families, their simple characteristics,
and how to combine them to create a more elaborate set of kernels. The properties of kernels
discussed here are well known in the literature. The purpose of this section is to bring them
together and illustrate the implications of different choices, as well as the structures that
can be achieved by combining them. These characteristics include additivity, symmetry,
periodicity, interactions between variables, and changepoints.

C.3.1 Families of kernels

In (single or multi-) kernel learning, kernels are the basic mechanism of conveying assump-
tions about the underlying functions we are interested in modeling. By selecting an individual
kernel, such as a square-exponential kernel (SE), we make certain assumptions about the
function, such as infinitely many derivatives in the case of the SE kernel. The advantage that
MKL brings lies in the combination of several local kernels, each similar to or different to each
other. Each type of kernel has its own set of assumptions about the smoothness of functions,
allowing MKL to take advantage of the collective strengths of several kernels and adapt to
a variety of data patterns. In essence, kernels are equipped with specific parameters that
control their functional forms. The selection and combination of kernels, along with the tuning
of their hyperparameters, can have an impact on the effectiveness of the model. Within these
families, there is a particular class of kernels that is of great importance.
C. Kernels
209

Universal kernels In [313], the authors describe a universal kernel as the kernel that has
a RKHS rich enough to arbitrarily well approximate any continuous function in a compact
metric space.

Definition 9. Universal kernel[314] A kernel k is called universal if the Hilbert space (RKHS)
Hk associated with K is dense in the space C(X) of continuous functions in a compact
metric space X . This means that for any function f ∈ C(X) and any ε > 0, there exists a
function g ∈ HK such that |f − g|∞ < ε where | · |∞ denotes the supremum norm.

In this definition, Hk being dense in C(X ) means that Hk is dense in C(X ) with respect
to the supremum norm. This implies that for any f ∈ C(X ), we can find a sequence of
functions gn ∈ Hk that converges to f uniformly. C(X) refers to the space of continuous
functions on the set X . Here, X is assumed to be a compact metric space. A metric space
(X , d) is a set X together with a metric or distance function d that allows measuring distances
between elements of X . The compactness of X means that it is closed and bounded.
Micchelli et al.[315] show that the universality of a kernel is equivalent to the universality of
its feature map Φ : X → H (where H is the Hilbert space). For example, the Gaussian RBF
|0x−y|02 i(x−y)
−
kernel K(x, y) = e 2σ 2 has a feature map Φ(x)(y) = e σ that is universal, since it
generates complex exponentials that are dense in C(Z). Many common kernels, such as dot
product, polynomial, and radial basis kernels, have been shown to be universal [315]. This
property makes universal kernels useful in machine learning algorithms such as SVMs.
210 C.3. Expressing structure through kernels

Figure C.1: Three plots are produced to demonstrate the universality property.
In each of the plots, the blue points represent the original data
, while the red lines express the function learned
by a kernel Ridge regression using the RBF kernel.
The RBF kernel captures the essence of various
data distributions,
from linear to sinusoidal to step functions.

Linear kernel The linear kernel is mathematically represented as

k(x, x′ ) = x⊤ x′ . (C.6)

Linear kernels are used to model linear functions. Given its nonstationary nature, the
predictions derived from this kernel are susceptible to change if the dataset undergoes any
shifts, all while keeping the kernel parameters constant.

Polynomial kernel The polynomial kernel can be expressed as:

k(x, x′ ) = (x⊤ x′ + c)d . (C.7)

Here, c and d are hyperparameters that represent he bias and the degree of the polynomial,
respectively. This kernel is adept at capturing non-linear patterns present in the data.
C. Kernels
211

Radial basis function kernel Also known as the squared exponential kernel, the Radial
Basis Function (RBF) kernel is denoted as:
!
∥x − x′ ∥2
k(x, x′ ) = exp − . (C.8)
2ℓ2

The hyperparameter ℓ defines the length scale and determines the width of the kernel and the
smoothness of the resultant function. This stationary kernel is designed to model functions
that are infinitely differentiable.

Figure C.2: These two images, demonstrate the kernel effect, using the corresponding relationship
between the raw data and the kernel matrix. We plot the stylized data on the left side and
a heatmap of the kernel values on the right. The darker shades indicate smaller values,
and the brighter shades indicate larger values. Points that are close together in the input
space have higher kernel values (brighter shades) than those that are farther apart.

Sigmoid kernel The sigmoid kernel is formulated as follows:

k(x, x′ ) = tanh(αx⊤ x′ + c). (C.9)

With α and c as hyperparameters. Although it is not positive definite, it has the capacity to
capture non-linear patterns in data.

Characteristic kernels Characteristic kernels have the following property: k(x, x) = 1.

Their primary objective is to quantify similarity, rather than emphasizing distance or magnitude.
Some illustrious examples include the cosine similarity kernel and normalized polynomial
212 C.3. Expressing structure through kernels

kernels.

C.3.2 Stationary versus non-stationary kernels

Next I discuss and compare stationary versus non-stationary kernels. A stationary kernel is
one which is translation-invariant [316], whereas nonstationary kernels are a general class of
kernels, which depend explicitly on the two input samples. We can use stationary kernels to
encode generic smoothness assumptions. On the other hand, with non-stationary kernels,
one can encode varying smoothness and correlations in the input space[316].

Stationary: k(x, x′ ) = k(∥x, x′ ∥).

Non-stationary: k(x, x′ ) = k(x, x′ ).

• Stationary kernels depend only on the distance between inputs ||x − x′ ||. This limits
their flexibility as the similarity between inputs is assumed constant across the input
space. Stationary kernels can impose constraints like symmetry and periodicity, hence
can be a simple way to incorporate smoothness and exploit geometric constraints.
Stationary kernels (like additive kernels) can propagate information far along axis
directions in the absence of data. This is also visible in the image C.3. Operators can
be applied to stationary kernels to impose symmetry and periodicity. This essentially is
a constraint on the posterior.

• Nonstationary kernels depend on the specific input locations x and x′ . This allows
modeling varying similarities and correlations across the input space. Whereas sta-
tionary kernels can face issues like over-/underestimation of uncertainty, these issues
can be mitigated with non-stationary kernels. Optimization is needed to estimate the
hyperparameters.
C. Kernels
213

Figure C.3: Representation of two Gramm matrices resulting from a stationary (RBF) and a non-
stationary (linear) kernel. Both functions show symmetry, although the stationary kernel
function stays the same along the diagonals, while the nonstationary kernel function does
not have this restriction hence being more intricate. The use of a (more) flexible inner
product within the space H leads to an adaptive measure of similarity.

C.3.3 Isotropic versus anisotropic kernels

When a kernel is dependent only on the norm of the lag vector between two examples, and
not on the direction, it is referred to as an isotropic (or homogeneous) kernel, meaning it
is only dependent on the distance between the kernel arguments, for example k(x, y) =
f (∥x − y∥), [316]. Now, let us provide a more clear definition. Let X be the input space. An
isotropic kernel kI : X × X → R satisfies the following: kI (x, x′ ) = kI (∥x − x′ ∥) ∀x, x′ ∈ X
where ∥ · ∥ is a norm on X , usually Euclidean distance. Some examples:

if ∥x − x′ ∥ ≥ θ

0,

kI (∥x − x′ ∥) = (C.10)
g(∥x − x′ ∥), x′ ∥


if ∥x − <θ

!
∥x − x′ ∥2 ′ ∥) = exp − ∥x − x∥
2
kI (∥x − x′ ∥) = 1 − kI (∥ x − x An isotropic stationary
∥x − x′ ∥2 + θ θ
′
kernel has spectral density SI (ω), with: kI (∥x − x ∥) =
intRd cos(ω T (x − z))SI (ω)
mathbf ω An anisotropic kernel kS : X × X → R depends explicitly on x, x′ : kS (x, z) ̸=
kS (∥x − z∥)
214 C.3. Expressing structure through kernels

∥x − x′ ∥2
kI ∥x − x′ ∥ = 1 −


(C.11)
∥x − x′ ∥2 + θ

and !
′
∥x − x′ ∥2
kI ∥x − x ∥ = exp − . (C.12)
θ

An isotropic stationary kernel is characterized by a spectral density SI (ω), where ω is

the frequency vector. The kernel is given by:

Z
′
cos(ω T (x − x′ ))SI (ω) dω.


kI ∥x − x ∥ = (C.13)
Rd

For anisotropic kernels, denoted by kS , the kernel depends explicitly on x and x′ and not
just on their difference:
kS (x, x′ ) ̸= kS ∥x − x′ ∥ .


(C.14)

An anisotropic stationary kernel has a general spectral density SS (ω1 , ω2 ), with ω1 and
ω2 as the frequency vectors. This relationship is represented by the following:

Z Z
kS (x, z) = cos(ω1T x − ω2T z)SS (ω1 , ω2 ) dω1 dω2 . (C.15)
Rd Rd

Isotropic kernels can only model radial effects, whereas anisotropic kernels can capture
directional relationships and more complex spatial structure.
C. Kernels
215

Figure C.4: These images demonstrate the concept of (an)-isotropy for kernels. The top 2 plots,
the isotropic Gaussian kernel is on the left side, which looks like a circular blur, and
the anisotropic Gaussian kernel is on the right, which looks more like an elliptical blur.
The difference in shape demonstrates their isotropy and anisotropy. The second set of
images visualizes an Isotropic Laplacian of Gaussian (LoG) kernel against an anisotropic
(Sobel) kernel. The term "isotropic" demonstrates that the kernel treats all directions
equally, which is evident from the symmetric pattern. The warm center indicates high
positive values, while the surrounding cooler colors represent the decreasing negative
values, forming a circular boundary that can detect edges in any direction. These
images representing the Sobel kernels are specifically designed to detect edges in
vertical and horizontal directions. The term anisotropic indicates that these kernels have
specific directional sensitivities. The left Sobel kernel (Sobel X) detects vertical edges by
responding to changes in brightness from left to right. The right Sobel kernel (Sobel Y) is
sensitive to horizontal edges by responding to brightness changes from top to bottom.
The color gradient (changing from cool to warm colors) indicates the range of values
within these kernels, with the warm colors highlighting the regions of highest positive
response and the cool colors indicating negative response.
216 C.3. Expressing structure through kernels

Figure C.5: Plots of commonly used isotropic stationary kernels. The left column displays the formula
for each kernel. The middle column depicts the kernel matrices, which represent the
pairwise kernel values between points. The right column presents the kernel functions
plotted against the distance between pairs of points, showing the kernel value as a
function of the distance.
C. Kernels
217

C.4 Overview of kernel learning

Kernel-based methods are widely used in modern machine learning frameworks to solve
various learning paradigms, such as classification, regression, and clustering. These methods
are considered basic tools in this field. The essence of these techniques is concentrated on
the integration of the functions enabled by the positive semi-defined kernel function. Within
the broadest mathematical framework, this specific function can be described as a scalar
product that operates in a Hilbert space.
We provide a general purpose definition:

Definition 10 (Kernel Learning). Kernel Learning algorithms refer to a family of algorithms

which capture complex nonlinear patterns by optimizing kernel functions tailored to each sce-
nario. These algorithms achieve efficient learning with infinite-dimensional linear models, with
runtime independent of feature space dimensionality, through kernel function optimization.

As mentioned previously, it is important to recognize that the selection of the incorrect

kernel inevitably leads to the selection of suboptimal features, affecting the effectiveness
of the learning process. To perform a classification task, we will utilize a family of kernel
machines that are a large-margin hyperplane classifier, known as support vector machines.
A support vector machine (SVM) is a method of supervised machine learning that allows
classification and regression based on structural risk minimization.
In general, the SVM receives a set of training samples x1 , . . . , xn with labels y1 , . . . , yn
to predict Yn+1 given Xn+1 . These methods make use of a hypothesis f such that f (Xn+1 )
is an approximation of Yn+1 . To achieve a reliable approximation, a loss function ℓ(y, z) is
associated with the risk of f measuring how different z is as a prediction of true y . We would
then like to choose a hypothesis that minimizes the expected risk, defined as the expectation
of the loss function.
def
ε(f ) = E [ℓ(Yn+1 , f (Xn+1 ))] . (C.16)

In most cases, the risk ε(f ) cannot be obtained since the joint distribution of Xn+1 , Yn+1 is
unknown. A common strategy is to choose the hypothesis that minimizes an estimation of
218 C.4. Overview of kernel learning

ε(f ) through the empirical risk.

n
1X
ε̂(f ) = ℓ(yk , f (xk )). (C.17)
n k=1

Under certain assumptions about the sequence of random variables xk , yk , if the set of
hypotheses is small enough, the minimizer of the empirical risk will closely approximate
the minimizer of the expected risk as n grows large. To have a well-defined solution to the
minimization problem, we must place constraints on the set H of hypotheses. If H is a
normed space (as is the case for an SVM), an effective technique is to consider only those
hypotheses f for which ∥f ∥H<k . This is equivalent to imposing a regularization penalty
R(f ) = λk ∥f ∥H , and solving the new optimization problem.

n
1X
min ε̂(f ) + R(f ) = min ℓ(yk , f (xk )) + λk ∥f ∥H , . (C.18)
f ∈F f ∈F n k=1

where λ is the degree of penalization, F is a class of decision functions, ℓ is the loss function,
and ∥f ∥H is a regularization functional. Among other classifiers, we consider the large-margin
classifier support vector machine which uses the hinge loss, or soft margin loss, defined as
[1 − yf (x)]+ .
C. Kernels
219

Figure C.6: The main objective of the support vector machine algorithm is to identify the optimal
hyperplane and efficiently distinguish the data points belonging to different classes. The
term "optimal" is used to refer to the hyperplane that shows the greatest margin between
the two classes, shown as blue and red dots in the diagram shown. SVMs differ from
other models in that they choose the optimal position of this plane, occupying the middle
point of the largest possible gap between the points of each class.

Therefore, the goal of SVM training is to perform the estimation or risk minimisation
previously outlined, which can be reformulated as a constrained optimisation objective.
The resulting decision that is optimized is then equivalent to determining a hyperplane of
separation with the maximum distance to the closest points of the identified classes. These
points are called support vectors. Taking into account a training set {(xi , yi )}N
i=1 , a feature

vector xi ∈ RD and class labels yi ∈ {−1, +1}, the hyperplane of separation can be defined
as d(xi , w, b) = w · xi + b = 0, where w ∈ RD represents the weight vector, b is a scalar
and the operation is a dot product. The optimal hyperplane that separates the data into two
220 C.4. Overview of kernel learning

classes minimises the objective function.

n
1X
max (0, 1 − yi (w · xi − b)) + λ∥w∥2. (C.19)
n i=1

This corresponds to a quadratic optimization problem and can be solved in the parameter
space with respect to w and b. There are several solutions to this problem, such as gradient
descent and coordinate descent methods. Starting from this primal form of optimization,
we next introduce slack variables where, for all i ∈ {1, . . . , n}, the optimizationles ξi =
max (0, 1 − yi (w · xi − b)), measure the distance ξi between a point and its crossing of the
margin. This relaxation allows one to accommodate a less than perfect linear separation of
training-set data. The optimization problem is given by the following:

n
1 X
minimize C ξi + λ|0w|02 . (C.20)
n i=1

subject to yi (w · xi − b) ≥ 1 − xi and ξi ≥ 0, for all i. The trade-off factor C compromises

between the maximization of the margin and the minimization of the misclassification error.
The primal problem is typically reformulated as a dual problem through a Lagrangian, and
the solution is guaranteed if the Karush-Kuhn-Tucker conditions [317] are verified. By solving
for the Lagrangian dual, the problem becomes

n n X n
X 1X
maximize f (α1 . . . αn ) = αi − yi αi (xi · xj )yj αj . (C.21)
i=1
2 i=1 j=1

Pn 1
subject to i=1 αi yi = 0, and 0 ≤ αi ≤ 2nλ ∀i. Since the dual maximisation problem
is a quadratic function of αi subject to linear constraints, it can be efficiently solved by
quadratic programming algorithms. Here, the variables αi are defined in such a way that
w = ni=1 αi yi xi . Furthermore, αi = 0 exactly when xi is on the correct side of the margin,
P

and 0 < αi < (2nλ)−1 when xi lies on the margin boundary. It follows that w can be written
as a linear combination of support vectors. The offset, b, can be recovered by finding a xi at
the margin boundary and solving yi (w · xi − b) = 1 ⇐⇒ b = w · xi − yi (note that yi−1 = yi
C. Kernels
221

since yi = ±1).

The presented framework provides a linear classifier assuming linear separability of the
data, which is, in practice, rare to observe. The solution to this problem is known as kernel
trick and extends such methods to nonlinear settings by projecting feature data xi ∈ X into a
transformed feature space through a nonlinear map ϕ(xi ) which, if selected adequately, will
provide close to perfect linear separability. This map ϕ : X → H is called the feature map,
H is the transformed feature space. In most cases, knowledge of this mapping is difficult to
select explicitly to achieve this objective, so it is common instead to utilize an implicit solution
to selecting this mapping by replacing it with a kernel representation.

Consider a kernel function defined in the original feature space k(xi , xj ) = ⟨ϕ(xi , Ψ), ϕ(xj , Ψ)⟩H .
It is selected to satisfy k : X × X → R and acts as an inner product representation in the
implicit separable Hilbert space H and the feature map ϕ : X → H so that ∀xi , xj ∈ X
we have k(·, ·) = ⟨ϕn (xi ), ϕn (xj )⟩H . The classification vector w in the transformed space
satisfies w = ni=1 αi yi ϕ(xi ) where αi is obtained by solving the maximization problem with
P

the cost function f given as follows

n n X n
X 1X
f (α1 . . . αn ) = αi − yi αi ⟨ϕ(xi ), ϕ(xj )⟩yj αj
i=1
2 i=1 j=1
n n X n (C.22)
X 1X
= αi − yi αi k(xi , xj )yj αj .
i=1
2 i=1 j=1

Pn 1
subject to i=1 αi yi = 0, and 0 ≤ αi ≤ 2nλ for all i. The coefficients αi can be solved using
quadratic programming, as before. Again, we can find some index i such that 0 < αi <
(2nλ)−1 , so that ϕ(xi ) lies on the boundary of the margin in the transformed space. Finally,
the optimal decision function of a classifier is

n
!
X
z 7→ sgn(w · ϕ(z) − b) = sgn αi yi k(xi , z) − b . (C.23)
i=1

Producing nonlinear classification decision boundaries dependent on the kernel choice and
222 C.5. Deep multiple kernel learning

kernel hyperparameters.

Generative feature spaces Features spaces are mathematical representations of data

that can be used to uncover hidden patterns and help with machine learning tasks, such as
classification, similarity assessment, or pattern recognition. Numerical representations can
be used to express characteristic values as a continuous space in R. Features can be simple,
like the presence of a horizontal line at a certain spot, or more intricate, like correlation. They
can be discrete (one of a finite set of values), binary (zero or one), or continuous.

Feature spaces are mathematical representations of data that seek to uncover any
underlying structure and make machine learning tasks, such as classification, easier. By
selecting a set of features from the data, a feature space is created which consists of all
possible values for those features. Feature values can be represented numerically, allowing
the feature space to be interpreted as a real space.

C.5 Deep multiple kernel learning

Deep multiple kernel learning (DMKL) aims to integrate the strengths of deep learning, which
is renowned for its capability in representation learning, and multiple kernel learning (MKL),
which offers a principled method for combining different views or characteristics of the data.
By combining these methodologies, DMKL enhances performance and flexibility compared
to conventional MKL.

C.5.1 Preliminaries and notation for DMKL

Let X denote the input space. Consider a collection of M base kernel functions {km : X × X → R}M
m=1

operating on X . Also, consider I as a finite index set with positive semidefinite kernels given
denoted by ki : X × X → R for i ∈ I .

Model set up Deep multiple kernel learning aims to learn hierarchical combinations of
kernels, similar to deep neural networks. The kernel at layer l is:
C. Kernels
223

 
Ml−1
kp(l) (x, x′ ) = σ  (x, x′ ) ,
X
(l) (l−1)
wqp kq (C.24)
q=1

(l)
where σ(·) is a nonlinear activation function that maintains positive semidefiniteness, wqp
are the combination weights and Ml−1 is the number of kernels in the previous layer l − 1.

The overall deep MKL optimization problem becomes the following:

n
1 (L) T (L) X
min w w +C ξi (C.25)
w(L) ,b(L) 2 i=1
w(l) ,θ (l)
 
(L) T (L) (L)
s.t. yi w ϕ (xi ) + b ≥ 1 − ξi , ξi ≥ 0, (C.26)

|0θ (l) |01 = 1, θp(l) ≥ 0, (C.27)

 
Ml−1
kp(l) (x, x′ ) (x, x′ ) ,
X
(l) (l−1)
= σ wqp kq (C.28)
q=1

here superscript l indicates the layer l and ϕ(L) (x) maps x to the feature space of the final
layer L. The problem is solved in a similar way by alternating optimization.

Deep multiple kernel learning framework

For l = 1, DMKL computes a combined kernel:

M
k1 (x, x′ ) = θ1,m km (x, x′ ),
X
(C.29)
m=1

where θ1 ∈ RM contains the kernel weights.

For the subsequent layers, nonlinear transformations ψl : X → Fl are applied to map

inputs into new feature spaces Fl . The kernel in the lth layer is:

Ml
kl (x, x′ ) = θl,m kl,m ψl−1 (x), ψl−1 (x′ ) .
X

(C.30)
m=1
224 C.5. Deep multiple kernel learning

Finally, a support vector machine (SVM) is trained on the last layer kernel kL for predictions,
training the entire architecture end-to-end via stochastic gradient descent. The kernel weights
θl,m and transformations ψl are learned together.

Comparing deep learning and MKL Deep neural network models excel in end-to-end
representation learning directly from raw data, however they require vast amounts of labeled
training data. Conversely, MKL depends on predefined kernels and doesn’t learn representa-
tions directly from raw data. However, it provides a principled approach for combining multiple
views of heterogeneous data. These two approaches can be combined, providing distinct
benefits:

• Hierarchical representation learning capabilities akin to deep neural networks.

• Interpretability via explicit kernel combinations.

• Avoidance of tuning individual kernel hyperparameters through end-to-end training.

Merging deep neural networks and multiple kernel learning, DMKL we can present an
innovative approach to representation learning using kernels. This can offer the best of both
worlds by combining the hierarchical feature learning capabilities of deep learning and the
principled kernel fusion approach of MKL.

Theoretical insights into kernel limitations

While the kernel trick affords remarkable flexibility in the construction of embedding spaces
that are suitable for linear methods, kernel-based learning is not without limitations stemming
from the implicit nature of the feature space. Both single kernel learning and multiple kernel
learning (MKL) approaches have theoretical benefits and limitations that merit analysis. In
this section, a brief overview of the core theoretical limitations will be presented.
First among these is the computational intractability of kernel methods for large datasets,
as the design matrix scales quadratically and cubically with sample size for prevalent tech-
niques like kernel SVMs. Consequently, the application of kernels is prohibitive for many big
C. Kernels
225

data problems without employing approximation schemes. However, the consequences of

such approximations on the learned decision boundary remain poorly characterized [318].
The kernel choice may enforce certain inductive biases and representations that may fail to
capture useful structure. Scaling single kernels to large datasets is difficult, though advances
in approximation methods help. Overall, single kernels can lack the adaptability and power
needed for some complex domains.

Further, the blessing and curse of kernels is the freedom to select the embedding
space. While kernels enable tailoring the feature space to the problem, insufficient theoretical
guidance exists for choosing an appropriate kernel. In practice, one resorts to cross-validation
over standard kernel families, hoping the true space aligns with one such class. Specifically, a
key limitation of single kernels is this dependence on the kernel choice. Picking an appropriate
kernel is vital yet principles for optimal selection are lacking [319]. Cross-validation over
standard kernels is common, hoping the true space aligns with one family. But better
manifolds may exist to disentangle the data [136, 320].

The geometry of the embedding space also remains entirely implicit due to the use of
kernel evaluations rather than explicit feature maps [136, 139]. As such, understanding the
decision boundary and extracted patterns becomes a challenge. While predictions may
improve, deeper understanding of the model’s functioning is unclear.

Finally, the flexibility of kernels comes at the cost of increased risk to overfit the finite
sample as one explores larger hypothesis spaces. Generalization ability relies strongly
on proper regularization [321–323]. However, precisely characterizing the variance and
regularization properties of a given kernel remains an open problem.

While MKL methods do provide greater modeling flexibility, they also have limitations.
Firstly, the MKL optimization is non-convex for general nonlinear combinations, unlike
SVMs [155]. This risks local optima. Global optimality is not guaranteed[324]. Secondly, MKL
has substantially higher computational costs, especially with numerous kernels [89, 325].

Another issue is regularization. Proper regularization is required to prevent overfitting but

little theory exists on optimal regularization in MKL [86]. Finally, interpreting MKL models
226 C.5. Deep multiple kernel learning

with many kernels is harder than single kernels [326]. Despite the benefits of MKL in terms
of representation, there are still issues with optimization, scalability, regularization, and
interpretation that need to be addressed. Further progress is necessary.

Shared Limitations Some limitations extend to both paradigms. kernel methods scale
poorly to large datasets as the design matrix grows quadratically. This makes big data
problems prohibitive without approximations. However, the impact of approximations on the
decision boundary is poorly understood. Also, the geometry of the embedding space remains
entirely implicit. This hinders understanding the decision boundary and learned patterns.
While predictions may improve, the interpretability of the model suffers.

Finally, greater flexibility increases overfitting risk. Generalization is strongly depen-

dent on proper regularization. But the precise characterization of a kernel’s variance and
regularization properties remains open [327].

Kernel learning as an alternative to deep learning? The generalization performance of

classifiers, especially in the context of deep learning, has gathered enough attention [328].
Deep learning models, which are inherently over-parametrized, have an interesting trait: they
seem to fit the training data perfectly, yet still generalize well to the test data in spite of the
apparent overfitting [329], [330]. This seemingly counterintuitive performance is not yet fully
theoretically understood.

Some researchers have suggested that kernel methods could serve as an alternative
paradigm to deep neural networks that may not suffer from such mysteries of generaliza-
tion [331]. Kernel learning approaches such as kernel SVM have well-defined regularization
properties and convergence guarantees that promote generalization, unlike deep overparam-
eterized models [332]. The implicit feature maps of the kernels also avoid the need to specify
network architectures.

However, challenges remain in scaling kernel methods to massive datasets and high-
dimensional data where deep learning excels [333]. The problem of choosing the optimal
kernel also persists, though multiple kernel learning helps by learning combinations of
C. Kernels
227

kernels [86]. Overall, while kernel methods have a strong theoretical grounding, they currently
lack the state-of-the-art performance of deep networks on complex real-world tasks involving
images, speech, and text [334].

Hybrid approaches that integrate deep representations with kernels may offer a promising
direction to balance interpretability and performance [335, 336]. At present, deep neural
networks are the most commonly used for most practical applications.

C.6 Inference procedures for time-series data: kernel two-sample

testing

Time series data present unique opportunities and challenges for statistical inference. Tradi-
tional statistical procedures and assumptions often do not apply directly to data that exhibit
temporal dependence and non-stationarity [259]. Thus, novel methods have been developed
specifically to make inferences on time series data.

Hence, we will present the notion of random variables X : E → R situated in a RKHS

and dive into the exploration of two-sample testing within the RKHS framework. Next, we
introduce the concept of maximum mean discrepancy (MMD) to differentiate between the
two probability distributions P and Q in a reproducing kernel Hilbert space. In this context,
the main aim is to examine the potential equivalence or differences between the distributions
of two populations or to evaluate the comparison of their respective characteristics, such as
moments, cumulants, and other relevant statistical properties. Therefore, the objective is to
examine the equivalence of two data sample distributions, characterized by {x1 , . . . , xn } and
{x′1 , . . . , x′n′ }∀i, j , with xi , x′j ∈ S .

The samples are independently generated from probability spaces (S, F, P) and (S, F, Q).
Our objective is to discern whether P = Q. In this case, xi and x′j represent individual time
series in the collections.
228 C.6. Inference procedures for time-series data: kernel two-sample testing

Two sample testing

One of the main research objectives of this thesis is to compare the distributions of feature
samples MFDFA-generated data collections. Specifically, our objective is to determine
whether two samples of multifractal detrended fluctuation analysis (MFDFA) data, denoted
P and Q, are generated according to the same distribution. The hypothesis testing for this
involves two distinct hypotheses:

H0 : P = Q, (C.31)

HA : P ̸= Q. (C.32)

H0 posits that the two samples originate from the same distribution, while HA is the com-
plement of H0 . To distinguish between these two hypotheses, we compute a test statistic
T based on sample data, which corresponds to sampling the true distribution of the test
statistic given finite data. The decision to reject or accept H0 is based on a predefined
threshold, typically the 95% quantile of the null distribution, denoted by t0.95 . If T falls below
this threshold, that is, P (T < t0.95 |H0 ), it is unlikely that the null distribution generated the
test statistic, leading to the rejection of H0 .
Our testing procedure aims for a criterion that takes a unique, positive value if P ̸= Q,
and zero if and only if P = Q. This process is associated with two types of errors:

1. Type I : this occurs when H0 is rejected incorrectly.

2. Type II : this occurs when HA is incorrectly accepted.

The level of a test, α, is a predetermined threshold denoting the maximum allowable

probability for a Type I error, i.e. incorrectly rejecting the null hypothesis H0 . The power of a
test, β , is the probability that it correctly rejects H0 when the alternative hypothesis HA is
true. A consistent test aims to achieve a level of α for the Type I error rate and reduce the
Type II error rate to zero as the number of samples approaches infinity.
C. Kernels
229

In practice, we often manually calculate the p-value, which is the quantile of the test
statistic in the null distribution, and compare it to the desired test power α, resulting in a
binary answer and setting an upper bound for a type I error. If α = 0.05, this implies a 5%
chance that the samples were generated under H0 , and hence sets an upper bound for the
probability of a type I error. Next, we describe a simple formulation of the problem:

Let x ∼ P and x′ ∼ Q be random variables, where P and Q are probability measures

corresponding to two different distributions on the real line. Let {xi }ni=1 and {x′j }m
j=1 be

independent and identically distributed (i.i.d.) samples drawn from the distributions P and Q
respectively, where each xi ∈ supp(P ) and each x′j ∈ supp(Q). We define the hypotheses
in Definition C.32. An ideal test is consistent with all alternatives. In other words, for any level
of non-zero significance α (the maximum allowable probability of falsely rejecting H0 and
hence an upper bound on the type-I error), as the sample sizes n, m → ∞, the probability of
correctly rejecting the null hypothesis when it is false (i.e. , the power of the test) approaches
1. This is written as:

Let n : sample size of sample X

Let m : sample size of sample Y

lim P(reject H0 |HA is true) = 1 .∀, α ∈ (0, 1] (C.33)

n,m→∞

To provide inference and overcome the limitation of simple summary statistics, we propose
using kernel methods that utilize entire data samples. A kernel k(x, x′ ), also known as a
covariance function, is a positive definite function with two inputs, x and x′ , which are vectors
in a Euclidean space Rp in our context.

C.6.1 Time series data embedding to kernel RKHS space

When constructing time series, one needs to define some sort of time series within the
sample. These will be used to make an inference or to seek empirical evidence to reject a
230 C.6. Inference procedures for time-series data: kernel two-sample testing

null statement that the data-generating population distributions for each time-series sample
set are equivalent.
In this context, one could map from time series data to summary statistics of each time
series data point and then seek statistical evidence that these features are sufficiently different
to reject a null that the two time series samples were from the same population. One could
resort to summary statistics such as the average value or variance. However, this might be
an overly simplistic measure for time series, as they can have rich features not captured
by these single summary statistics. The challenge with this approach is that the summary
statistics may not characterize sufficiently the features of the time-series distribution.
Rather than relying on simple summary statistics to test a hypothesis and disregarding the
dependence structure of time-series data, we make use of kernel methods. This approach is
more complex, as it requires a mechanism to measure the similarity between pairs of time
series of each population sample. To this end, we focus on computing the similarity between
two time series in a RKHS.

Maximum mean discrepancy Maximum mean discrepancy (MMD) is a powerful tool for
comparing two probability distributions P, Q within a RKHS. This statistical test is based
on the principle that two distributions are different if and only if there exists at least one
function that has a different expectation for each distribution. The foundation of MMD lies
in the theoretical aspects of RKHS, which provide a rich structure for the development of
algorithms and statistical tests. Per [14, 104], for any function f in the unit ball of a universal
RKHS H, we define empirical MMD as E[f (x)] − E[f (x′ )], where x ∼ P and x′ ∼ Q. This
discrepancy measure is essentially the difference in the means of the function f evaluated
in samples from P and Q, respectively [104], [337, 338]. Large values of this test statistic
suggest that time series samples likely originate from different distributions [105]. If the MMD
is zero, this indicates that the samples could have been drawn from the same distribution,
under the null hypothesis. Per [14], smoothness is enforced by restricting the witness function
to a unit ball in a RKHS In simple terms, when this statistic takes large values, the samples
are likely to come from different distributions. We show this with the help of an example in
C. Kernels
231

Figure C.7]. The two distributions in this example are Gaussian and Gumbel. We took 21,000
samples from each and used a Gaussian kernel to compute the MMD witness function. We
can see that the witness function is positive when the normal probability exceeds the Gumbel
probability and negative when the Gumbel probability exceeds the normal probability.

Figure C.7: As discussed, one of the interesting features of MMD is the ability to see the difference in
the distributions via sampling from their empirical witness function as shown here. Here,
we show the difference in the mean embeddings in the RKHS. The difference of the two
samples is shown by evaluating the witness function at various points. When computing
the maximum mean discrepancy (MMD) between two distributions and producing a
measure of global deviation between the distributions, the witness function is created that
detects when two samples diverge. Insofar as the smoothness constraint allows, the size
of the witness function is a function of how much one density exceeds the other. This is
an obvious result, and it is easy to see why such a function would maximize the mean
disparity in the linked RKHS. Furthermore, the witness function does not generate an
interpretable result for distributions whose characteristic functions display local changes.
It does not provide a direct indication of the distribution properties that, when compared,
would maximize test power; instead, test power is determined by the witness function
norm and (generally speaking) its variance under the null determines test power.

In practical applications, MMD provides a mechanism to measure the similarity between

pairs of time-series data from each population sample. We use kernel functions to map the
time series data to the RKHS H, which is a high-dimensional or infinite-dimensional space
where the data could be more easily separated. The similarity between two time series
data X and X ′ is calculated by calculating their inner product in the RKHS H, denoted as
232 C.6. Inference procedures for time-series data: kernel two-sample testing

⟨X, X ′ ⟩H . This inner product in H essentially serves as a measure of the similarity between
the two time series.

In the context of time series analysis, the sample space S typically includes all possible
time series that can occur, that is, S = {x1 , x2 , . . . , x|X | } where each xi represents a time
series. The σ -algebra F , which is the collection of all possible events, is defined as the power
set of S , denoted as F = P (S). Given this, the probability function P : F → [0, 1] determines
the likelihood of obtaining a particular time series in the set of time series data samples. As
an example, we could define a population distribution to be uniform P (xi (|xi |)) = 1/|M|,
where M = |S|2 is the total number of possible time series and xi (|xi |) is a time series with
|xi | number of time points. The premise of this statistic is that, when this statistic is applied to
samples from various distributions, it will determine how likely these distributions will differ.
We now provide a more precise definition:

Definition 11. Maximum mean discrepancy [339] For appropriate kernels, MMD can be
defined as a distance on P2 (X). The MMD can be written as the RKHS norm of the
unnormalised witness function fµ,ν between µ and ν , which is the difference between the
mean embeddings of ν and µ, The MMD can be thought of as either,


MMD(P, Q) = sup EX∼P [f (X)]
f ∈H:∥f ∥H ≤1

(C.34)
− EY ∼Q [f (Y )]


MMD(P, Q) = sup E [f (X)]
f ∈H:∥f ∥H ≤1 X∼P
 (C.35)
− E [f (Y )]
Y ∼Q

or, more relevantly right now,

MMD[F, p, q] = ||µp − µq ||2F MMD[F, p, q] = ∥µp − µq ∥2F (C.36)

where φ : X → H is the feature map. Note that this formulation does not assume any form
C. Kernels
233

of input data.

Linear and quadratic MMD are two variants of MMD, distinguished by the complexity of
their computations and the insights they provide. Linear MMD is computationally efficient
and straightforward and is widely used for simple distribution comparison tasks. However,
quadratic MMD, though computationally more intensive, provides richer information by consid-
ering interactions between pairs of samples, making it suitable for more complex tasks. Below
is a detailed overview of these two algorithms, description of their strengths, and comparison.

Linear MMD Let p and q be two probability distributions with domains X and let F be
a RKHS of real-valued functions on X . The linear maximum mean discrepancy (MMD) is
defined as:

MMDl2 (p, q) = sup f ∈ F, , |f |F ≤ 1 (Ex ∼ p[f (x)] − Ex ∼ q[f (x)])2 , (C.37)

where | · |F denotes the norm in F . Linear MMD measures the distance between the mean
embeddings of p and q in F .

Quadratic time MMD The quadratic MMD is defined as follows. Let p and q be two
probability distributions with domains X . Let F be an RKHS of functions on X and let
k(x, x′ ) = ⟨ϕ(x), ϕ(x′ )⟩H be the associated kernel function, where ϕ : X → H is a feature
map in another RKHS H. Then the quadratic MMD is defined as:

MMDq 2 (p, q) = Ex, x′ ∼ p[k(x, x′ )]+Ey, y ′ ∼ q[k(y, y ′ )]−2Ex ∼ p, y ∼ q[k(x, y)], (C.38)

where Ex, x′ ∼ p[k(x, x′ )] is the expectation of the kernel function k under the joint distri-
bution of p and Ey, y ′ ∼ q[k(y, y ′ )] is the expectation of k under the joint distribution of q .
The term 2Ex∼p,y∼q [k(x, y)] measures the covariance between p and q . The quadratic-time
MMD is essentially the squared difference of the sample means if we had used a linear
kernel (where the feature-space embedding is the identity function). We will use the unbiased
234 C.6. Inference procedures for time-series data: kernel two-sample testing

statistic here, defined as:

n X m n X m n X m
1 X 1 X 1 X
k(xi , xj ) + k(yi , yj ) − 2 k(xi , yj )
n(n − 1) i=1 j̸=i m(m − 1) i=1 j̸=i nm i=1 j=1

One feature of MMD is that through visualization, we can determine where the density
functions are different by querying the (empirical) MMD witness function. This is simply
the difference between the empirical mean embeddings in the RKHS and therefore can be
computed (recall that the MMD is the RKHS norm of this witness function). Estimation of the
p-value of the test can be addressed by sampling.

Figure C.8: In [14], have created a method for creating a kernel-based witness function. In this
example, the witness function that identifies when two samples diverge is a byproduct
of calculating the maximum mean discrepancy (MMD) between two distributions and
calculating a measure of global deviation between the distributions.

f ∗ (·) = µ̂X (·) − µ̂Y (·)

Estimating expected values by averaging independent samples yields an unbiased estimate

C. Kernels
235

for MMD samples.

m X m n X n m X n
1 X 1 X 2 X
MMDu [F, X, Y ]2 = k(xi , xj )+ k(yi , yj )− k(xi , yj )
m(m − 1) i=1 j̸=i n(n − 1) i=1 j̸=i mn i=1 j̸=i
(C.39)
A biased estimate would be

m X m n X n m X n
1 X 1 X 2 X
MMDb [F, X, Y ]2 = k(x i , xj )+ k(yi , yj )− k(xi , yj ) (C.40)
m2 i=1 j=1 n2 i=1 j=1 mn i=1 j̸=i

Differences between linear and quadratic MMD As discussed in the previous section,
MMD assesses the dissimilarity of two probability distributions by comparing their respective
feature expectations in a RKHS. MMD comes in two varieties: linear MMD and quadratic
MMD. While both versions of MMD utilize kernel functions to compare distributions, they
compute the kernel matrix differently. Linear MMD employs a kernel matrix derived as the
inner product of the feature vectors of the data points from the two distributions. Linear MMD
has been shown to equal the squared distance between the empirical means of the two
distributions in feature space [14].

Therefore, linear MMD is computationally efficient and simple to implement. Nevertheless,

it may not reflect intricate interactions between distributions, particularly if the distributions
have nonlinear characteristics. In contrast, quadratic MMD employs a quadratic kernel
matrix generated as the product of two linear kernel matrices. The first linear kernel matrix
is computed as the inner product of the feature vectors of the data points from the first
distribution, whereas the second linear kernel matrix is computed as the inner product of the
feature vectors from the second distribution.

Linear MMD may not be able to capture nonlinear interactions between distributions,
whereas quadratic MMD can. However, quadratic MMD is computationally more costly than
linear MMD due to the additional computation of the kernel matrix. Whereas the linear MMD
measures the distance between the mean embeddings of p and q in F , the quadratic MMD
measures the distance between the distributions themselves in the RKHS induced by the
kernel function. Linear MMD is a special case of quadratic MMD, where the feature map ϕ
236 C.6. Inference procedures for time-series data: kernel two-sample testing

is chosen to be the identity map. In general, the quadratic MMD is more powerful than the
linear MMD, as it can detect differences between distributions that cannot be detected by
the linear MMD. We use quadratic MMD for all of our experiments. Finally, MMD, like many
statistical tests, is associated with the concepts of type I and type II errors, which originate
from classical inference procedures as described earlier.
Bibliography

[1] R. Velasco-Fuentes and W. L. Ng, “Nonlinearities in Stochastic Clocks: Trades and

Volume as Subordinators of Electronic Markets,” Quantitative Finance, vol. 11, no. 6,
pp. 863–881, 2011. (pages 8, 28, 31, 43, 44, 99, and 152)

[2] M. O’Hara, “High frequency market microstructure,” Journal of Financial Economics,

vol. 116, no. 2, pp. 257–270, 2015. (page 8)

[3] D. Easley, M. M. L. de Prado, and M. O’Hara, “The Volume Clock: Insights into the
High-Frequency Paradigm,” Journal of Portfolio Management, vol. 39, no. 1, pp. 19–29,
2012. (pages 43, 56, 94, 99, 100, 119, and 122)

[4] M. Kearns and Y. Nevmyvaka, “Machine learning for market microstructure and high
frequency trading Maureen O’Hara (Author, Editor), Marcos Lopez de Prado (Editor),
David Easley (Editor) Risk books,” High-Frequency Trading–New Realities for Traders,
Markets and Regulators, pp. 91–124, 2013. (pages 8 and 120)

[5] H. Ali, F. Aslam, and P. Ferreira, “Modeling dynamic multifractal efficiency of US

electricity market,” Energies, vol. 14, oct 2021. (page 8)

[6] S. Dutta, D. Ghosh, and S. Samanta, “Multifractal detrended cross-correlation analysis

of gold price and SENSEX,” Physica A: Statistical Mechanics and its Applications,
vol. 413, pp. 195–204, nov 2014.

[7] J. Gui, Z. Zheng, D. Fu, Y. Fu, and Z. Liu, “Long-term correlations and multifractality of
toll-free calls in China,” Physica A: Statistical Mechanics and its Applications, vol. 567,
apr 2021. (page 8)

237
238 Bibliography

[8] R. Cont, “Empirical properties of asset returns: Stylized facts and statistical issues,”
Quantitative Finance, vol. 1, pp. 223–236, feb 2001.
(pages 28, 55, 118, 150, and 154)

[9] J. P. Bouchaud, M. Mézard, and M. Potters, “Statistical Properties of Stock Order

Books: Empirical Results and Models,” Quantitative Finance, vol. 2, pp. 251–256, aug
2002. (pages 28, 51, and 56)

[10] L. Peng, “Learning with information capacity constraints,” Journal of Financial and
Quantitative Analysis, vol. 40, pp. 307–329, jun 2005. (page 28)

[11] R. Jennings and L. Starks, “Information Content and the Speed of Stock Price Adjust-
ment,” Journal of Accounting Research, vol. 23, no. 1, p. 336, 1985. (page 28)

[12] J. Hasbrouck, “One Security, Many Markets: Determining the Contributions to Price
Discovery,” The Journal of Finance, vol. 50, pp. 1175–1199, sep 1995. (page 28)

[13] M. D. Gould, M. A. Porter, S. Williams, M. McDonald, D. J. Fenn, and S. D. Howison,

“Limit order books,” Quantitative Finance, vol. 13, pp. 1709–1742, nov 2013.
(pages 28 and 188)

[14] A. Gretton, B. Sriperumbudur, D. Sejdinovic, H. Strathmann, S. Balakrishnan, M. Pontil,

and K. Fukumizu, “Optimal kernel choice for large-scale two-sample tests,” Advances
in Neural Information Processing Systems, vol. 2, pp. 1205–1213, 2012.
(pages 28, 40, 51, 52, 158, 230, 234, and 235)

[15] J. E. Griffin and R. C. Oomen, “Sampling returns for realized variance calculations:
Tick time or transaction time?,” Econometric Reviews, vol. 27, no. 1-3, pp. 230–253,
2008. (pages 28 and 150)

[16] K. C. Lee and S. B. Oh, “An intelligent approach to time series identification by a
neural network-driven decision tree classifier,” Decision Support Systems, vol. 17,
pp. 183–197, jul 1996. (page 29)
Bibliography
239

[17] M. Bicego, V. Murino, and M. A. Figueiredo, “Similarity-based classification of se-

quences using hidden Markov models,” Pattern Recognition, vol. 37, no. 12, pp. 2281–
2291, 2004. (page 29)

[18] S. Saeed and H. C. Ong, “Performance of SVM with multiple kernel learning for classi-
fication tasks of imbalanced datasets,” Pertanika Journal of Science and Technology,
vol. 27, no. 1, pp. 527–545, 2019. (pages 30 and 49)

[19] V. López, A. Fernández, S. García, V. Palade, and F. Herrera, “An insight into
classification with imbalanced data: Empirical results and current trends on using
data intrinsic characteristics,” Information Sciences, vol. 250, pp. 113–141, 2013.
(pages 30, 49, and 101)

[20] F. McGroarty, A. Booth, E. Gerding, and V. L. Chinthalapati, “High frequency trading

strategies, market fragility and price spikes: an agent based model perspective,” Annals
of Operations Research, vol. 282, no. 1-2, pp. 217–244, 2019. (page 31)

[21] S. Huang and B. Z. Yueshen, “Speed acquisition,” Management Science, vol. 67,
pp. 3492–3518, jun 2021. (page 31)

[22] A. Wehrli and D. Sornette, “The excess volatility puzzle explained by financial noise
amplification from endogenous feedbacks,” Scientific Reports, vol. 12, nov 2022.
(page 31)

[23] K. Xu, M. D. Gould, and S. D. Howison, “Multi-Level Order-Flow Imbalance in a Limit

Order Book,” Market Microstructure and Liquidity, vol. 04, no. 03n04, p. 1950011, 2018.
(page 31)

[24] K. al Dayri, E. Bacry, and J. F. Muzy, “Econophysics of order-driven markets,” 2011.

(page 31)

[25] M. Bicego, E. Grosso, and E. Otranto, “Recognizing and forecasting the sign of financial
local trends using Hidden Markov Models,” Centre for North South Economic Research,
University of Cagliari and University of Sassari, Sardinia, 2008. (page 34)
240 Bibliography

[26] J. P. Bouchaud, J. D. Farmer, and F. Lillo, “How Markets Slowly Digest Changes in
Supply and Demand,” in Handbook of Financial Markets: Dynamics and Evolution,
pp. 57–160, Elsevier Inc., 2009. (pages 40, 41, and 172)

[27] F. Pérez-Cruz and O. Bousquet, “Kernel methods and their potential use in signal
processing,” IEEE Signal Processing Magazine, vol. 21, no. 3, pp. 57–65, 2004.
(pages 40 and 45)

[28] A. Gretton, O. Bousquet, A. Smola, and B. Schlkopf, “Measuring statistical dependence

with Hilbert-Schmidt norms,” Lecture Notes in Computer Science (including subseries
Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 3734
LNAI, pp. 63–77, 2005. (pages 40, 51, and 153)

[29] J. W. Kantelhardt, E. Koscielny-Bunde, H. H. Rego, S. Havlin, and A. Bunde, “Detecting

long-range correlations with detrended fluctuation analysis,” Physica A: Statistical
Mechanics and its Applications, vol. 295, no. 3-4, pp. 441–454, 2001.
(pages 40, 50, 52, 151, and 156)

[30] L. Dall’Amico, A. Fosset, J. P. Bouchaud, and M. Benzaquen, “How does latent liquidity
get revealed in the limit order book?,” Journal of Statistical Mechanics: Theory and
Experiment, vol. 2019, jan 2019. (page 41)

[31] H. Beltran-Lopez, J. Grammig, and A. J. Menkveld, “Limit order books and trade
informativeness,” The European Journal of Finance, vol. 18, pp. 737–759, oct 2012.
(page 41)

[32] T. G. Andersen, M. Thyrsgaard, and V. Todorov, “Time-Varying Periodicity in Intraday

Volatility,” Journal of the American Statistical Association, vol. 114, pp. 1695–1707, oct
2019. (page 41)

[33] T. Chen, K. C. Chan, and H. Chang, “Periodicity of trading activity in foreign exchange
markets,” Journal of Financial Research, vol. 45, pp. 445–465, jun 2022. (page 42)
Bibliography
241

[34] M. C. Tseng and S. Mahmoodzadeh, “Information Jumps, Liquidity Jumps, and Market
Efficiency,” Journal of Risk and Financial Management, vol. 15, mar 2022. (page 42)

[35] D. Easley, R. F. Engle, M. O’hara, and L. Wu, “Time-varying arrival rates of informed
and uninformed trades,” Journal of Financial Econometrics, vol. 6, no. 2, pp. 171–207,
2008. (pages 42 and 99)

[36] P. K. Clark, “A Subordinated Stochastic Process Model with Finite Variance for Specu-
lative Prices,” Econometrica, vol. 41, no. 1, p. 135, 1973. (page 42)

[37] U. A. Müller, M. M. Dacorogna, R. B. Olsen, O. V. Pictet, M. Schwarz, and C. Mor-

genegg, “Statistical study of foreign exchange rates, empirical evidence of a price
change scaling law, and intraday analysis,” Journal of Banking and Finance, vol. 14,
no. 6, pp. 1189–1208, 1990. (pages 43 and 57)

[38] H. Geman, D. Madan, and M. Yor, “Asset Prices Are Brownian Motion: Only in Business
Time,” in Quantitative Analysis in Financial Markets, pp. 103–146, World Scientific,
2001. (page 43)

[39] T. Ané and H. Geman, “Order flow, transaction clock, and normality of asset returns,”
Oct. 2000. (page 56)

[40] C. Marinelli, S. T. Rachev, R. Roll, and H. Göppl, “Subordinated Stock Price Models:
Heavy Tails and Long-Range Dependence in the High-frequency Deutsche Bank Price
Record,” in Datamining and Computational Finance, pp. 69–94, Springer, 2000.

[41] A. Türkoğlu, “Normally distributed high-frequency returns: a subordination approach,”

Quantitative Finance, vol. 16, pp. 389–409, mar 2016.

[42] W. Bounliphone, E. Belilovsky, M. B. Blaschko, I. Antonoglou, and A. Gretton, “A test

of relative similarity for model selection in generative models,” in 4th International
Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings,
2016. (page 43)
242 Bibliography

[43] R. C. Oomen, “High-dimensional covariance forecasting for short intra-day horizons,”

Quantitative Finance, vol. 10, no. 10, pp. 1173–1185, 2010. (page 43)

[44] B. Mandelbrot, “The Variation of Certain Speculative Prices,” in Fractals And Scaling In
Finance, pp. 39–78, Springer, 2015. (pages 43 and 50)

[45] C. C. Elgot, “Structured Programming With and Without GO TO Statements,” IEEE

Transactions on Software Engineering, vol. SE-2, no. 1, pp. 41–54, 1976. (page 43)

[46] M. O’Hara, “High-frequency trading and its impact on markets,” Financial Analysts
Journal, vol. 70, no. 3, pp. 18–27, 2014. (pages 43, 119, 120, and 122)

[47] D. Easley, N. M. Kiefer, and M. O’Hara, “The information content of the trading process,”
Journal of Empirical Finance, vol. 4, no. 2-3, pp. 159–186, 1997. (pages 43 and 152)

[48] P. Gopikrishnan, V. Plerou, Y. Liu, L. A. Amaral, X. Gabaix, and H. E. Stanley, “Scaling

and correlation in financial time series,” Physica A: Statistical Mechanics and its
Applications, vol. 287, pp. 362–373, dec 2000. (page 43)

[49] H. Geman, “Stochastic Clock and Financial Markets,” in Special Volume: Mathematical
Modeling and Numerical Methods in Finance (Bensoussan, A and Zhang, Q, ed.),
vol. 15 of Handbook of Numerical Analysis, pp. 649–663, North Holland, 2009.
(pages 43, 94, and 99)

[50] M. E. Holder, M. Qi, and A. K. Sinha, “The impact of time duration between trades on
the price of treasury note futures contracts,” Journal of Futures Markets, vol. 24, no. 10,
pp. 965–980, 2004. (page 44)

[51] N. Huth and F. Abergel, “The times change: multivariate subordination. Empirical facts,”
Quantitative Finance, vol. 12, pp. 1–10, jan 2012. (page 44)

[52] S. J. Hardimana, N. Bercot, and J. P. Bouchaud, “Critical reflexivity in financial markets:

A Hawkes process analysis,” European Physical Journal B, vol. 86, oct 2013.
(pages 44, 57, and 152)
Bibliography
243

[53] D. Easley, M. M. López De Prado, and M. O’Hara, “Flow toxicity and liquidity in a
high-frequency world,” Review of Financial Studies, vol. 25, pp. 1457–1493, may 2012.
(pages 44, 94, 100, and 102)

[54] B. M. Henrique, V. A. Sobreiro, and H. Kimura, “Literature review: Machine learning

techniques applied to financial market prediction,” Expert Systems with Applications,
vol. 124, pp. 226–251, jun 2019. (page 45)

[55] W. Huang, Y. Nakamori, and S. Y. Wang, “Forecasting stock market movement direction
with support vector machine,” Computers and Operations Research, vol. 32, no. 10,
pp. 2513–2522, 2005. (page 45)

[56] K. J. Kim, “Financial time series forecasting using support vector machines,” Neuro-
computing, vol. 55, pp. 307–319, sep 2003.

[57] T. Fletcher, Z. Hussain, and J. Shawe-Taylor, “Currency Forecasting using Multiple

Kernel Learning with Financially Motivated Features,” Networks, pp. 1–6, 2010.

[58] T. Fletcher, Z. Hussain, and J. Shawe-Taylor, “Multiple Kernel Learning on the Limit
Order Book,” Quantitative Finance, vol. 00, no. 00, p. 0, 2011. (pages 47 and 122)

[59] Y. Guo, S. Zhang, and Y. Liu, “Research on Risk Features and Prediction of China’s
Crude Oil Futures Market Based on Machine Learning,” Frontiers in Energy Research,
vol. 10, jul 2022. (page 45)

[60] F. E. H. Tay and L. Cao, “Application of support vector machines in financial time series
forecasting,” Omega, vol. 29, no. 4, pp. 309–317, 2001. (page 45)

[61] U. Thissen, R. Van Brakel, A. P. De Weijer, W. J. Melssen, and L. M. Buydens, “Using

support vector machines for time series prediction,” Chemometrics and Intelligent
Laboratory Systems, vol. 69, no. 1-2, pp. 35–49, 2003. (page 45)

[62] A. N. Kercheval and Y. Zhang, “Modelling high-frequency limit order book dynamics
244 Bibliography

with support vector machines,” Quantitative Finance, vol. 15, no. 8, pp. 1315–1329,
2015. (pages 45, 46, 122, and 123)

[63] F. E. H. Tay and L. J. Cao, “A comparative study of saliency analysis and genetic
algorithm for feature selection in support vector machines,” Intelligent Data Analysis,
vol. 5, no. 3, pp. 191–209, 2001. (pages 46 and 122)

[64] T. Van Gestel, J. A. Suykens, D. E. Baestaens, A. Lambrechts, G. Lanckriet, B. Van-

daele, B. De Moor, and J. Vandewalle, “Financial time series prediction using least
squares support vector machines within the evidence framework,” IEEE Transactions
on Neural Networks, vol. 12, no. 4, pp. 809–821, 2001.

[65] H. Yang, L. Chan, and I. King, “Support vector machine regression for volatile stock
market prediction,” in Lecture Notes in Computer Science (including subseries Lecture
Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 2412 of IDEAL
’02, (London, UK), pp. 391–396, Springer-Verlag, 2002.

[66] W. Huang, Y. Nakamori, and S. Y. Wang, “Forecasting stock market movement direction
with support vector machine,” Computers and Operations Research, vol. 32, no. 10,
pp. 2513–2522, 2005. (pages 46 and 122)

[67] T. Fletcher, Z. Hussain, and J. Shawe-Taylor, “Multiple Kernel Learning on the Limit
Order Book.,” Journal of Machine Learning Research-Proceedings Track, vol. 11,
pp. 167–174, 2010. (pages 46, 49, and 123)

[68] Á. Cartea and S. Jaimungal, “Modelling Asset Prices for Algorithmic and High-
Frequency Trading,” Applied Mathematical Finance, vol. 20, pp. 512–547, dec 2013.
(pages 46, 47, 118, 120, 121, 122, 125, and 128)

[69] Á. Cartea and T. Meyer-Brandis, “How duration between trades of underlying securities
affects option prices,” Review of Finance, vol. 14, pp. 749–785, oct 2010. (page 47)
Bibliography
245

[70] Á. Cartea and D. Karyampas, “Volatility and covariation of financial assets: A high-
frequency analysis,” Journal of Banking and Finance, vol. 35, pp. 3319–3334, dec
2011. (page 46)

[71] R. Azencott, A. Beri, Y. Gadhyan, N. Joseph, C. A. Lehalle, and M. Rowley, “Real-time

market microstructure analysis: online transaction cost analysis,” Quantitative Finance,
vol. 14, pp. 1167–1185, jul 2014. (page 46)

[72] J. A. Lasserre, C. M. Bishop, and T. P. Minka, “Principled hybrids of generative and

discriminative models,” in 2006 IEEE Computer Society Conference on Computer
Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 87–94, IEEE, 2006.
(pages 47, 65, 66, 118, and 122)

[73] T. Fletcher and J. Shawe-Taylor, “Multiple Kernel Learning with Fisher Kernels for High
Frequency Currency Prediction,” Computational Economics, vol. 42, pp. 1–24, aug
2013. (pages 47, 118, and 122)

[74] R. El-Yaniv and D. Pidan, “Selective prediction of financial trends with hidden markov
models,” in Proceedings of the 24th International Conference on Neural Information
Processing Systems, NIPS’11, (Red Hook, NY, USA), p. 855–863, Curran Associates
Inc., 2011. (page 47)

[75] J. T. Halloran and D. M. Rocke, “Gradients of generative models for improved discrimi-
native analysis of tandem mass spectra,” in Advances in Neural Information Processing
Systems (I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan,
and R. Garnett, eds.), vol. 30, Curran Associates, Inc., 2017. (page 47)

[76] I. Visser, M. E. J. Raijmakers, and P. C. M. Molenaar, “Confidence intervals for hidden

markov model parameters,” British Journal of Mathematical and Statistical Psychology,
vol. 53, no. 2, pp. 317–327, 2000. (page 47)

[77] M. Bicego, E. Pȩkalska, D. M. Tax, and R. P. Duin, “Component-based discriminative

246 Bibliography

classification for hidden Markov models,” Pattern Recognition, vol. 42, pp. 2637–2648,
nov 2009. (pages 47, 118, and 121)

[78] M. Bicego, A. Ulaş, U. Castellani, A. Perina, V. Murino, A. F. Martins, P. M. Aguiar, and

M. A. Figueiredo, “Combining information theoretic kernels with generative embeddings
for classification,” Neurocomputing, vol. 101, pp. 161–169, 2013.
(pages 47, 118, 121, and 122)

[79] J. Lasserre, “Hybrids of Generative and Discriminative Methods for Machine Learning,”
Philosophy, no. March, 2008. (pages 47 and 65)

[80] M. F. Valstar and M. Pantic, “Combined support vector machines and hidden markov
models for modeling facial action temporal dynamics,” in Human-Computer Interaction
(M. Lew, N. Sebe, T. Huang, and E. Bakker, eds.), vol. 4796 of Lecture Notes in
Computer Science, pp. 118–127, Berlin: Springer, 2007. IEEE International Workshop
on Human - Computer Interaction. (pages 47 and 122)

[81] K. H. Brodersen, T. M. Schofield, A. P. Leff, C. S. Ong, E. I. Lomakina, J. M. Buhmann,

and K. E. Stephan, “Generative embedding for model-based classification of fMRI
data,” PLoS computational biology, vol. 7, no. 6, p. e1002079, 2011. (page 47)

[82] J. Mager, U. Paasche, and B. Sick, “Forecasting financial time series with support
vector machines based on dynamic kernels,” in SMCia/08 - Proceedings of the 2008
IEEE Conference on Soft Computing on Industrial Applications, pp. 252–257, 2008.
(page 48)

[83] T. Azim, “Fisher kernels match deep models,” Electronic Letters, vol. 53, MAR 16 2017.
(pages 48 and 182)

[84] M. Sewell and J. Shawe-Taylor, “Forecasting foreign exchange rates using kernel
methods,” Expert Systems with Applications, vol. 39, no. 9, pp. 7652–7662, 2012.
(page 48)
Bibliography
247

[85] G. Montana and F. Parrella, “Learning to trade with incremental support vector re-
gression experts,” in Lecture Notes in Computer Science (including subseries Lecture
Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5271 LNAI,
pp. 591–598, Springer, 2008. (page 48)

[86] M. Gonen and E. Alpaydin, “Multiple kernel learning algorithms,” Journal of Machine
Learning Research, vol. 12, no. Jul, pp. 2211–2268, 2011. (pages 48, 225, and 227)

[87] X. Gong, B. Zou, Y. Duan, J. Xu, Q. Luo, and Y. Yang, “Multiple Kernel SVM Based on
Two-Stage Learning,” IEEE Access, vol. 8, pp. 101133–101144, 2020. (page 48)

[88] V. N. Vapnik, “An overview of statistical learning theory,” IEEE Transactions on Neural
Networks, vol. 10, no. 5, pp. 988–999, 1999. (page 48)

[89] F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan, “Multiple kernel learning, conic duality,
and the smo algorithm,” in Proceedings of the Twenty-First International Conference
on Machine Learning, ICML ’04, (New York, USA), p. 6, Association for Computing
Machinery, 2004. (pages 49 and 225)

[90] Z. Hussain and J. Shawe-Taylor, “Improved loss bounds for multiple kernel learning,”
in International Conference on Artificial Intelligence and Statistics (Geoffrey Gordon
David Dunson and M. Dudk, eds.), vol. 15, (Brookline, Mass, USA), pp. 370–377,
Microtome Publisher, 2011.

[91] F. Aiolli and M. Donini, “Easymkl: a scalable multiple kernel learning algorithm,” Neuro-
computing, vol. 169, pp. 215–224, Dec. 2015. Bruges, Belgium. (pages 49 and 72)

[92] T. Fletcher, Z. Hussain, and J. Shawe-Taylor, “{M}ultiple {K}ernel {L}earning on the

{L}imit {O}rder {B}ook,” Quantitative Finance -To be published, vol. 00, no. 00, p. 0,
2011. (page 49)

[93] T. Fletcher, Machine learning for financial market prediction. PhD thesis, University
College London, 2012. (page 49)
248 Bibliography

[94] G. Oh, S. Kim, and C. Eom, “Long-term memory and volatility clustering in high-
frequency price changes,” Physica A: Statistical Mechanics and its Applications,
vol. 387, no. 5-6, pp. 1247–1254, 2008. (page 50)

[95] L. Pérez-Sienes, M. Grande, J. C. Losada, and J. Borondo, “The Hurst Exponent as

an Indicator to Anticipate Agricultural Commodity Prices,” Entropy, vol. 25, apr 2023.
(page 50)

[96] J. Barunik, T. Aste, T. Di Matteo, and R. Liu, “Understanding the source of multifractality
in financial markets,” Physica A: Statistical Mechanics and its Applications, vol. 391,
pp. 4234–4251, sep 2012. (pages 50 and 153)

[97] S. Wanqing, X. Chen, C. Cattani, and E. Zio, “Multifractional Brownian Motion and
Quantum-Behaved Partial Swarm Optimization for Bearing Degradation Forecasting,”
Complexity, vol. 2020, jan 2020. (page 50)

[98] T. Liu and J. Lemeire, “Effective and Efficient Identification of Persistent-state Hidden
(semi-) Markov Models,” 2014. (page 50)

[99] C. W. J. Granger and R. Joyeux, “An introduction to long-memory time series models
and fractional differencing,” Journal of Time Series Analysis, vol. 1, no. 1, pp. 15–29,
1980. (page 50)

[100] M. D. Gould, M. A. Porter, and S. D. Howison, “The Long Memory of Order Flow
in the Foreign Exchange Spot Market,” Market Microstructure and Liquidity, vol. 02,
p. 1650001, jun 2016. (pages 50 and 168)

[101] F. Ren, G. F. Gu, and W. X. Zhou, “Scaling and memory in the return intervals of
realized volatility,” Physica A: Statistical Mechanics and its Applications, vol. 388,
pp. 4787–4796, nov 2009. (pages 50 and 56)

[102] K. Chwialkowski, A. Ramdas, D. Sejdinovic, and A. Gretton, “Fast two-sample testing

with analytic representations of probability measures,” Advances in Neural Information
Processing Systems, vol. 2015-Janua, pp. 1981–1989, 2015. (pages 51 and 153)
Bibliography
249

[103] W. Jitkrittum, W. Xu, Z. Szabo, K. Fukumizu, and A. Gretton, “A Linear-Time Kernel

Goodness-of-Fit Test,” in Advances in Neural Information Processing Systems (Guyon,
I and Luxburg, UV and Bengio, S and Wallach, H and Fergus, R and Vishwanathan, S
and Garnett, R, ed.), vol. 30 of Advances in Neural Information Processing Systems,
2017. (pages 51, 52, 158, and 169)

[104] A. Schrab, I. Kim, M. Albert, B. Laurent, B. Guedj, and A. Gretton, “MMD Aggregated
Two-Sample Test,” Arxiv, 2021. (pages 52 and 230)

[105] K. M. Borgwardt, A. Gretton, M. J. Rasch, H. P. Kriegel, B. Schölkopf, and A. J.

Smola, “Integrating structured biological data by Kernel Maximum Mean Discrepancy,”
Bioinformatics, vol. 22, no. 14, pp. 49–57, 2006. (pages 52, 153, and 230)

[106] W. Huang, K. Tan, Z. Zhang, J. Hu, and S. Dong, “A Review of Fusion Methods for
Omics and Imaging Data,” IEEE/ACM Transactions on Computational Biology and
Bioinformatics, vol. 20, no. 1, pp. 74–93, 2023. (pages 52 and 153)

[107] H. Zhao, X. Yang, B. Chen, H. Chen, and W. Deng, “Bearing fault diagnosis using
transfer learning and optimized deep belief network,” Measurement Science and
Technology, vol. 33, jun 2022. (pages 52 and 153)

[108] G. Wang, B. Zhao, B. Wu, C. Zhang, and W. Liu, “Intelligent prediction of slope stability
based on visual exploratory data analysis of 77 in situ cases,” International Journal of
Mining Science and Technology, vol. 33, pp. 47–59, jan 2022. (pages 52 and 153)

[109] K. Chwialkowski, H. Strathmann, and A. Gretton, “A kernel test of goodness of fit,” 33rd
International Conference on Machine Learning, ICML 2016, vol. 6, pp. 3854–3867,
2016. (pages 52 and 156)

[110] W. Jitkrittum, A. Gretton, N. Heess, S. M. Eslami, B. Lakshminarayanan, D. Sejdinovic,

and Z. Szabó, “Kernel-based just-in-time learning for passing expectation propagation
messages,” in Uncertainty in Artificial Intelligence - Proceedings of the 31st Conference,
UAI 2015 (Meila, M and Heskes, T, ed.), pp. 405–414, 2015. (page 52)
250 Bibliography

[111] J. W. Kantelhardt, S. A. Zschiegner, E. Koscielny-Bunde, S. Havlin, A. Bunde, and

H. E. Stanley, “Multifractal detrended fluctuation analysis of nonstationary time series,”
Physica A: Statistical Mechanics and its Applications, vol. 316, pp. 87–114, dec 2002.
(pages 52, 88, 151, and 156)

[112] E. A. Ihlen, “Introduction to multifractal detrended fluctuation analysis in Matlab,” Fron-

tiers in Physiology, vol. 3 JUN, 2012. (pages 52 and 150)

[113] Z. G. Yu, V. Anh, and R. Eastes, “Multifractal analysis of solar flare indices and their
horizontal visibility graphs,” Nonlinear Processes in Geophysics, vol. 19, no. 6, pp. 657–
665, 2012. (pages 52 and 150)

[114] Z.-Q. Jiang, W.-J. Xie, W.-X. Zhou, and D. Sornette, “Multifractal Analysis of Financial
Markets: A Review,” Reports on Progress of Physics, vol. 82, p. 125901, dec 2019.
(pages 52, 53, 151, and 175)

[115] S. Begušić, Z. Kostanjčar, H. Eugene Stanley, and B. Podobnik, “Scaling properties of

extreme price fluctuations in Bitcoin markets,” Physica A: Statistical Mechanics and its
Applications, vol. 510, pp. 400–406, nov 2018. (pages 52 and 152)

[116] M. Shu and W. Zhu, “Real-time prediction of Bitcoin bubble crashes,” Physica A:
Statistical Mechanics and its Applications, vol. 548, jun 2020. (page 53)

[117] S. Liu, W. Fang, X. Gao, F. An, M. Jiang, and Y. Li, “Long-term memory dynamics of
crude oil price spread in non-dollar countries under the influence of exchange rates,”
Energy, vol. 182, pp. 753–764, 2019. (page 53)

[118] Y. S. Feng and H. Y. Wang, “Multifractal Fluctuation Analysis of Correlations between

the Sector Stock Markets in China and the US,” Fluctuation and Noise Letters, vol. 20,
aug 2021. (page 53)

[119] A. Dmitriev, A. Lebedev, V. Kornilov, and V. Dmitriev, “Multifractal Early Warning Signals
about Sudden Changes in the Stock Exchange States,” Complexity, vol. 2022, sep
2022. (page 53)
Bibliography
251

[120] X. Li, M. Sun, C. Gao, D. Han, and M. Wang, “The parametric modified limited pen-
etrable visibility graph for constructing complex networks from time series,” Physica
A: Statistical Mechanics and its Applications, vol. 492, pp. 1097–1106, feb 2018.
(page 53)

[121] Y.-S. Feng, Y. Li, and B.-M. Cao, “Asymmetric multifractal cross-correlations between
economic policy uncertainty and agricultural futures prices,” Fluctuation And Noise
Letters, vol. 21, aug 2022. (page 53)

[122] J. Wang, W. Jiang, Y. Yan, W. Shao, X. Wu, and Z. Hua, “Exploring the asymmetric
multifractal characteristics of price-volume cross-correlation in the Chinese rebar
futures market based on MF-ADCCA,” Fluctuation and Noise Letters, vol. 22, jun 2023.
(page 54)

[123] T. Takaishi, “Time Evolution of Market Efficiency and Multifractality of the Japanese
Stock Market,” Journal of Risk and Financial Management, vol. 15, jan 2022. (page 54)

[124] M. A. Naeem, E. Bouri, Z. Peng, S. J. H. Shahzad, and X. V. Vo, “Asymmetric effi-

ciency of cryptocurrencies during COVID19,” Physica A: Statistical Mechanics and its
Applications, vol. 565, mar 2021. (page 54)

[125] M. Buchanan, “It’s a (stylized) fact!,” Nature Physics, vol. 8, no. 1, p. 3, 2011. (page 55)

[126] C. Schinckus, “Stylized facts and multiple realizability in econophysics,” Results in

Physics, vol. 4, pp. 135–136, 2014. (pages 55 and 154)

[127] L. Gillemot, J. D. Farmer, and F. Lillo, “There’s more to volatility than volume,” Quantita-
tive Finance, vol. 6, no. 5, pp. 371–384, 2006. (page 55)

[128] J.-P. Bouchaud, Y. Gefen, M. Potters, and M. Wyart, “Fluctuations and response in
financial markets: the subtle nature of ‘random’ price changes,” Quantitative Finance,
vol. 4, pp. 176–190, apr 2004. (pages 56, 152, and 166)
252 Bibliography

[129] K. Matia, Y. Ashkenazy, and H. E. Stanley, “Multifractal properties of price fluctuations

of stocks and commodities,” Europhysics Letters, vol. 61, pp. 422–428, feb 2003.
(pages 56 and 152)

[130] M. I. Krivoruchenko, E. Alessio, V. Frappietro, and L. J. Streckert, “Modeling stylized

facts for financial time series,” [Link], 2004. (page 57)

[131] G. A. Christodoulakis, “Common volatility and correlation clustering in asset returns,”

European journal of operational research, vol. 182, no. 3, pp. 1263–1284, 2007.
(page 57)

[132] M. Shakeel and B. Srivastava, “Stylized facts of high-frequency financial time series
data,” Global business review, vol. 22, no. 2, pp. 550–564, 2021. (page 57)

[133] Y. Bengio, L. Yao, G. Alain, and P. Vincent, “Generalized denoising auto-encoders as

generative models,” in Advances in Neural Information Processing Systems, pp. 899–
907, 2013. (page 61)

[134] R. Caruana, “Multitask learning,” Machine learning, vol. 28, no. 1, pp. 41–75, 1997.
(pages 62 and 67)

[135] B. Scholkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio,

“Toward Causal Representation Learning,” Proceedings of the IEEE, vol. 109, pp. 612–
634, may 2021. (page 62)

[136] B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, “Estimating

the support of a high-dimensional distribution,” Neural Computation, vol. 13, pp. 1443–
1471, jul 2001. (pages 62 and 225)

[137] B. Schölkopf, R. Herbrich, and A. J. Smola, “A generalized representer theorem,” in

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial
Intelligence and Lecture Notes in Bioinformatics) (D. Helmbold and B. Williamson,
eds.), vol. 2111 of Lecture Notes in Artifical Inellgience, pp. 416–426, Natl Res Inst
Bibliography
253

Math Comp Sci; Amsterdam Hist Museum; Netherlands Org Sci Res, 2001.
(pages 62 and 206)

[138] J. Liu, B. Li, and W. S. Zhang, “Feature extraction using maximum variance sparse
mapping,” Neural Computing and Applications, vol. 21, no. 8, pp. 1827–1833, 2012.
(page 62)

[139] B. Schlkopf, A. J. Smola, and F. Bach, Learning with Kernels: Support Vector Machines,
Regularization, Optimization, and Beyond. The MIT Press, 2018.
(pages 62, 63, 203, 207, and 225)

[140] N. Cristianini and J. Shawe-Taylor, An introduction to support vector machines and

other kernel-based learning methods. Cambridge University Press, 2000.
(pages 62, 63, and 207)

[141] M. Belkin, P. Niyogi, and V. Sindhwani, “Manifold regularization: A geometric framework

for learning from labeled and unlabeled examples,” Journal of Machine Learning
Research, vol. 7, pp. 2399–2434, 2006. (page 62)

[142] M. Tanveer, S. Sharma, R. Rastogi, and P. Anand, “Sparse support vector machine
with pinball loss,” Transactions on Emerging Telecommunications Technologies, vol. 32,
no. 2, 2021. (page 63)

[143] J. Shawe-Taylor and N. Cristianini, Kernel methods for pattern analysis. Cambridge
university press, 2004. (pages 63 and 118)

[144] S. Saitoh, “A reproducing kernel theory with some general applications,” in Springer
Proceedings in Mathematics and Statistics (T. Qian and L. G. Rodino, eds.), vol. 177
of Springer Proceedings in Mathematics & Statistics, (New York, USA), pp. 151–182,
ISAAC, Springer, 2016. (page 63)

[145] N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola, “On kernel-target align-

ment,” in Advances in Neural Information Processing Systems (T. Dietterich, S. Becker,
254 Bibliography

and Z. Ghahramani, eds.), vol. 14, pp. 367–373, 2002. 15th Annual Conference on
Neural Information Processing Systems (NIPS), VANCOUVER, CANADA, DEC 03-08,
2001. (page 64)

[146] G. Bouchard, W. Triggs, and B. Triggs, “The tradeoff between generative and discrimi-
native classifiers,” in 16th IASC International Symposium On Computational Statistics
(Compstat’04), pp. 721–728, 2004. (page 64)

[147] C. M. Bishop, Pattern Recognition and Machine Learning, vol. 16 of Information

Science and Statistics. New Jersey, USA: Springer-Verlag New York, Inc., 2007.
(pages 65 and 191)

[148] G. E. Dahl, D. Yu, L. Deng, and A. Acero, “Context-dependent pre-trained deep

neural networks for large-vocabulary speech recognition,” IEEE Transactions on Audio,
Speech and Language Processing, vol. 20, pp. 30–42, jan 2012. (page 67)

[149] C. Li, M. Georgiopoulos, and G. C. Anagnostopoulos, “A unifying framework for typical

multitask multiple kernel learning problems,” IEEE Transactions on Neural Networks
and Learning Systems, vol. 25, no. 7, pp. 1287–1297, 2014. (page 67)

[150] L. A. Bull, P. Gardner, T. J. Rogers, E. J. Cross, N. Dervilis, and K. Worden, “Proba-

bilistic Inference for Structural Health Monitoring: New Modes of Learning from Data,”
ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil
Engineering, vol. 7, no. 1, 2021. (page 67)

[151] C. Li, M. Georgiopoulos, and G. C. Anagnostopoulos, “Pareto-path multitask multiple

kernel learning,” IEEE Transactions on Neural Networks and Learning Systems, vol. 26,
no. 1, pp. 51–61, 2015. (page 67)

[152] T. Wang, H. Su, and J. Li, “DWS-MKL: Depth-width-scaling multiple kernel learning for
data classification,” Neurocomputing, vol. 411, pp. 455–467, oct 2020. (page 72)

[153] C. Cortes, M. Mohri, and A. Rostamizadeh, “Algorithms for learning kernels based on
Bibliography
255

centered alignment,” Journal of Machine Learning Research, vol. 13, pp. 795–828,
2012. (page 79)

[154] N. Cristianini, J. Kandola, A. Elisseeff, and J. Shawe-Taylor, “On kernel target align-
ment,” in Studies in Fuzziness and Soft Computing, vol. 194, pp. 205–256, Berlin/Hei-
delberg: Springer-Verlag, 2006. (page 79)

[155] G. R. Lanckriet, N. Cristianini, P. Bartlett, L. El Ghaoui, and M. I. Jordan, “Learning the

kernel matrix with semidefinite programming,” Journal of Machine Learning Research,
vol. 5, pp. 27–72, 2004. (pages 80 and 225)

[156] M. López de Prado, Advances in Financial Machine Learning: Lecture 9/10. John
Wiley & Sons, 2018. (pages 82, 94, and 101)

[157] J. Beran, Statistics for long-memory processes, vol. 61 of Monographs on statistics

and applied probability. Boca Raton, USA: Chapman Hall, 1994. (pages 83 and 85)

[158] H. Yan, J. S. Chan, and G. W. Peters, “Long memory models for financial time series
of counts and evidence of systematic market participant trading behaviour patterns in
futures on US Treasuries,” SSRN Electronic Journal, 2017. (page 85)

[159] M. A. Delgado and C. Velasco, “Sign tests for long-memory time series,” Journal of
Econometrics, vol. 128, no. 2, pp. 215–251, 2005. (page 85)

[160] J. G. De Gooijer and R. J. Hyndman, “25 years of time series forecasting,” International
Journal of Forecasting, vol. 22, no. 3, pp. 443–473, 2006. Twenty five years of
forecasting. (page 90)

[161] J. Hasbrouck and G. Saar, “Low-latency trading,” Journal of Financial Markets, vol. 16,
no. 4, pp. 646–679, 2013. (page 94)

[162] R. Cont, “Statistical modeling of high-frequency financial data,” IEEE Signal Processing
Magazine, vol. 28, pp. 16–25, sep 2011. (pages 94 and 150)
256 Bibliography

[163] F. Abergel, M. Anane, A. Chakraborti, A. Jedidi, and I. M. Toke, Physics of society:

Econophysics and sociophysics: Limit order books. Cambridge: Cambridge University
Press, 2016. (page 94)

[164] K. H. Bae and P. Dixon, “Do investors use options and futures to trade on different
types of information? Evidence from an aggregate stock index,” Journal of Futures
Markets, vol. 38, pp. 175–198, feb 2018. (pages 94 and 101)

[165] L. Van Der Maaten and G. Hinton, “Visualizing data using t-SNE,” Journal of Machine
Learning Research, vol. 9, pp. 2579–2625, 2008. (pages 94, 95, 96, and 97)

[166] P. Hoffman, G. Grinstein, K. Marx, I. Grosse, and E. Stanley, “DNA visual and analytic
data mining,” in Proceedings of the IEEE Visualization Conference, pp. 437–441, IEEE,
1997. (pages 94, 95, and 97)

[167] M. C. Wang, L. P. Zu, and C. J. Kuo, “The state of the electronic limit order book, order
aggressiveness and price formation,” Asia-Pacific Journal of Financial Studies, vol. 37,
pp. 245–296, apr 2008. (pages 94 and 107)

[168] T. Zhang, G. F. Gu, H. C. Xu, X. Xiong, W. Chen, and W. X. Zhou, “Power-law tails in the
distribution of order imbalance,” Physica A: Statistical Mechanics and its Applications,
vol. 483, pp. 201–208, oct 2017. (pages 94, 107, 152, and 153)

[169] L. Liu and Z. Wang, “Encoding Temporal Markov Dynamics in Graph for Visualizing
and Mining Time Series,” Arxiv, oct 2016. (page 95)

[170] M. Dredze, P. P. Talukdar, and K. Crammer, “Sequence Learning from Data with Multiple
Labels,” in ECML/PKDD Workshop on Learning from Multi-Label Data, no. Figure 1,
p. 39, 2009. (page 96)

[171] B. Frénay and M. Verleysen, “Classification in the presence of label noise: A survey,”
IEEE Transactions on Neural Networks and Learning Systems, vol. 25, pp. 845–869,
may 2014. (page 95)
Bibliography
257

[172] D. Huang, J. Unnikrishnan, S. Meyn, V. Veeravalli, and A. Surana, “Statistical SVMs for
robust detection, supervised learning, and universal classification,” in Proceedings -
2009 IEEE Information Theory Workshop on Networking and Information Theory, ITW
2009, pp. 62–66, IEEE, 2009. (page 95)

[173] S. Chakrabartty and G. Cauwenberghs, “Forward decoding kernel machines: A hy-

brid HMM/SVM approach to sequence recognition,” in Lecture Notes in Computer
Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes
in Bioinformatics), vol. 2388, pp. 278–292, Springer, 2002. (pages 95 and 96)

[174] M. Balamurali, “t-Distributed Stochastic Neighbor Embedding.” Webpage, 2022.

(page 96)

[175] P. Hoffman, G. Grinstein, and D. Pinkney, “Dimensional anchors: A graphic primitive

for multidimensional multivariate information visualizations,” in Proceedings of the
1999 Workshop on New Paradigms in Information Visualization and Manipulation in
conjunction with the 8th ACM Internation Conference on Information and Knowledge
Management, NPIVM 1999 (C. D. Shaw and D. S. Ebert, eds.), NPIVM, (New York, NY),
pp. 9–16, ACM International Conference on Information and Knowledge Management,
Association for Computing Machinery, 1999. (page 97)

[176] L. Novakova and O. Stpankova, “Visualization of trends using radviz,” Journal of

intelligent information systems, vol. 37, no. 3, pp. 355–369, 2011. (page 97)

[177] D. Probst and J. L. Reymond, “Visualization of very large high-dimensional data sets
as minimum spanning trees,” Journal of Cheminformatics, vol. 12, feb 2020. (page 97)

[178] J. R. Black, P. K. Jain, and W. Sun, “Trade-time clustering,” Review of Quantitative

Finance and Accounting, vol. 60, no. 3, pp. 1209–1242, 2023. (page 99)

[179] J. Kang, K. Y. Kwon, and W. Kim, “Flow toxicity of high-frequency trading and its impact
on price volatility: Evidence from the KOSPI 200 futures market,” Journal of Futures
Markets, vol. 40, pp. 164–191, feb 2020. (page 100)
258 Bibliography

[180] J. Sirignano and R. Cont, “Universal features of price formation in financial markets:
perspectives from deep learning,” Quantitative Finance, vol. 19, pp. 1449–1459, sep
2019. (pages 101 and 120)

[181] T. Fischer and C. Krauss, “Deep learning with long short-term memory networks for
financial market predictions,” European Journal of Operational Research, vol. 270,
no. 2, pp. 654–669, 2018. (page 101)

[182] Q. Liu, J. Zhang, and H. Zhang, “Eliminating the redundant source effects from the
cross-correlation reverse-time migration using a modified stabilized division,” Comput-
ers and Geosciences, vol. 92, pp. 49–57, jul 2016. (page 107)

[183] D. Makowiec, B. Graff, A. Kaczkowska, G. Graff, D. Wejer, J. Wdowczyk, M. Żarczyńska-

Buchowiecka, M. Gruchała, and Z. R. Struzik, “Visualization of short-term heart period
variability with network tools as a method for quantifying autonomic drive,” ECG
Time Series Variability Analysis: Engineering and Medicine, pp. 141–158, jul 2017.
(page 107)

[184] J. D. Hamilton, “Analysis of time series subject to changes in regime,” Journal of

Econometrics, vol. 45, no. 1-2, pp. 39–70, 1990. (page 118)

[185] P. C. Phillips and Z. Xiao, “A primer on unit root testing,” Journal of Economic Surveys,
vol. 12, no. 5, pp. 423–470, 1998.

[186] A. Ang and A. Timmermann, “Regime changes and financial markets,” Annual Review
of Financial Economics, vol. 4, no. 1, pp. 313–337, 2012. (page 118)

[187] L. Bauwens and D. Veredas, “The stochastic conditional duration model: A latent
variable model for the analysis of financial durations,” Journal of Econometrics, vol. 119,
no. 2, pp. 381–412, 2004. (page 118)

[188] J. M. Vanden, “Equilibrium analysis of volatility clustering,” Journal of Empirical Finance,

vol. 12, no. 3, pp. 374–417, 2005. (page 118)
Bibliography
259

[189] O. E. Barndorff-Nielsen, P. R. Hansen, A. Lunde, and N. Shephard, “Realized kernels

in practice: Trades and quotes,” Econometrics Journal, vol. 12, no. 3, pp. C1–C32,
2009.

[190] E. M. Aldrich, I. Heckenbach, and G. Laughlin, “A compound duration model for high-
frequency asset returns,” Journal of Empirical Finance, vol. 39, pp. 105–128, dec 2016.
(pages 119 and 121)

[191] Q. Wang, B. Teng, Q. Hao, and Y. Shi, “High-frequency Statistical Arbitrage Strategy
Based on Stationarized Order Flow Imbalance,” in Procedia Computer Science (Y. Sun,
P. Thomas, R. Bie, and X. Cheng, eds.), vol. 187 of Procedia Computer Science, (SARA
BURGERHARTSTRAAT 25, PO BOX 211, 1000 AE AMSTERDAM, NETHERLANDS),
pp. 518–523, ELSEVIER SCIENCE BV, 2021. (pages 118, 120, and 121)

[192] Bernhard Schölkopf and A. J. Smola, Learning with Kernels Support, vol. 62. Citeseer,
Jan 2002. (page 118)

[193] M. F. Valstar and M. Pantic, “Combined support vector machines and hidden markov
models for modeling facial action temporal dynamics - Abstract,” in Belgian/Netherlands
Artificial Intelligence Conference, pp. 367–368, Springer, 2008. (page 118)

[194] S. Wang, Q. Huang, S. Jiang, and Q. Tian, “S 3 MKL: Scalable semi-supervised multiple
kernel learning for real-world image applications,” IEEE Transactions on Multimedia,
vol. 14, pp. 1259–1274, aug 2012. (pages 118 and 151)

[195] R. Zhang and C. K. Ho, “MIMO broadcasting for simultaneous wireless information
and power transfer,” IEEE Transactions on Wireless Communications, vol. 12, no. 5,
pp. 1989–2001, 2013. (page 118)

[196] T. G. Andersen, “Return volatility and trading volume: An information flow interpretation
of stochastic volatility,” Journal of Finance, vol. 51, pp. 169–204, mar 1996.
(pages 118 and 125)
260 Bibliography

[197] M. M. Dacorogna, An Introduction to High-Frequency Finance. Academic Press, 2001.

(page 118)

[198] P. Holmes and M. Tomsett, “Information and noise in U.K. futures markets,” Journal of
Futures Markets, vol. 24, pp. 711–731, aug 2004. (page 118)

[199] P. Pasquariello and C. Vega, “Informed and strategic order flow in the bond markets,”
Review of Financial Studies, vol. 20, pp. 1975–2019, nov 2007. (page 118)

[200] A. P. Chaboud, B. Chiquoine, E. Hjalmarsson, and C. Vega, “Rise of the machines:

Algorithmic trading in the foreign exchange market,” Journal of Finance, vol. 69, no. 5,
pp. 2045–2084, 2014. (pages 118 and 130)

[201] D. Easley, M. Lopez De Prado, and M. O’hara, “Discerning information from trade data,”
Journal of Financial Economics, vol. 120, pp. 269–285, 2016. (page 119)

[202] H. Berkman and P. D. Koch, “Noise trading and the price formation process,” Journal of
Empirical Finance, vol. 15, pp. 232–250, mar 2008. (page 119)

[203] D. P. Murphy and R. S. Thirumalai, “Short-Term Return Predictability and Repetitive

Institutional Net Order Activity,” Journal of Financial Research, vol. 40, no. 4, pp. 455–
477, 2017. (page 119)

[204] S. C. Andrade, C. Chang, and M. S. Seasholes, “Trading imbalances, predictable

reversals, and cross-stock price pressure,” Journal of Financial Economics, vol. 88,
pp. 406–423, may 2008. (page 119)

[205] S. Sohn and X. Zhang, “Could the Extended Trading of CSI 300 Index Futures Facilitate
Its Role of Price Discovery?,” Journal of Futures Markets, vol. 37, pp. 717–740, jul
2017. (page 119)

[206] P. Wu and W. Siwasarit, “Capturing the Order Imbalance with Hidden Markov Model: A
Case of SET50 and KOSPI50,” Asia-Pacific Financial Markets, vol. 27, pp. 115–144,
mar 2020. (page 119)
Bibliography
261

[207] W. X. Li, C. C. S. Chen, and J. Nguyen, “Which market dominates the price discovery
in currency futures? The case of the Chicago Mercantile Exchange and the Interconti-
nental Exchange,” Global Finance Journal, vol. 52, may 2022. (pages 120 and 121)

[208] J. A. Sirignano, “Deep learning for limit order books,” Quantitative Finance, vol. 19,
pp. 549–570, apr 2019. (page 120)

[209] J. Pohle, R. Langrock, F. M. van Beest, and N. M. Schmidt, “Selecting the number of
states in hidden Markov models: pragmatic solutions illustrated using animal move-
ment,” Journal of agricultural, biological, and environmental statistics, vol. 22, no. 3,
pp. 270–293, 2017. (page 121)

[210] M. Nasir and C. Ezeife, I, “Semantic enhanced markov model for sequential e-
commerce product recommendation,” International Journal of Data Science and Ana-
lytics, vol. 15, pp. 67–91, JAN 2023. (page 121)

[211] R. F. Engle and J. R. Russell, “Autoregressive Conditional Duration: A New Model

for Irregularly Spaced Transaction Data,” Econometrica, vol. 66, p. 1127, sep 1998.
(page 121)

[212] M. Bicego, E. Grosso, and E. Otranto, “A hidden Markov model approach to classify
and predict the sign of financial local trends,” in Lecture Notes in Computer Science
(including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioin-
formatics) (K. T. G. M. R. F. K. J. A. G. C. L. da Vitoria Lobo N., ed.), vol. 5342 LNCS,
pp. 852–861, Berlin Heidelberg: Springer, dec 2008. (page 121)

[213] T. Jaakkola and D. Haussler, “Exploiting generative models in discriminative classifiers,”

in Proceedings of the 1998 conference (M. I. Jordan, M. J. Kearns, and S. A. Solla,
eds.), vol. 11 of Advances in neural information processing systems, (Cambridge,
Mass), pp. 487–493, MIT Press, 1999. (pages 121 and 122)

[214] A. Carli, M. Bicegoy, S. Baldo, and V. Murinoy, “Nonlinear mappings for generative
kernels on latent variable models,” in Proceedings - International Conference on
262 Bibliography

Pattern Recognition, (New Jersey, USA), pp. 2134–2137, IEEE Computer Society,
2010. (page 121)

[215] L. Chen, H. Man, and A. V. Nefian, “Face recognition based on multi-class mapping of
Fisher scores,” Pattern Recognition, vol. 38, no. 6, pp. 799–811, 2005. (page 122)

[216] Z. Ghahramani, “An introduction to hidden Markov models and Bayesian networks,”
2001. (page 122)

[217] R. Bertolini and S. J. Finch, “Stability of filter feature selection methods in data pipelines:
a simulation study,” International Journal of Data Science and Analytics, 2022 DEC 14
2022. (page 124)

[218] A. Kirilenko, A. S. Kyle, M. Samadi, and T. Tuzun, “The Flash Crash: High-Frequency
Trading in an Electronic Market,” Journal of Finance, vol. 72, pp. 967–998, jun 2017.
(page 125)

[219] E. Rengifo and R. Trendafilov, “High-frequency trading under information regimes,” in

Handbook of High Frequency Trading (G. N. Gregoriou, ed.), pp. 279–303, London,
UK: Academic Press, 2015.

[220] D. M. Rothschild and R. Sethi, “Trading Strategies and Market Microstructure: Evidence
from a Prediction Market,” The Journal of Prediction Markets, vol. 10, no. 1, pp. 1–29,
2016. (page 125)

[221] V. Filimonov and D. Sornette, “Apparent criticality and calibration issues in the Hawkes
self-excited point process model: application to high-frequency financial data,” Quanti-
tative Finance, vol. 15, no. 8, pp. 1293–1314, 2015. (page 128)

[222] M. Dixon, D. Klabjan, and J. H. Bang, “Classification-based financial markets prediction

using deep neural networks,” Algorithmic Finance, vol. 6, no. 3-4, pp. 67–77, 2017.
(page 140)
Bibliography
263

[223] M. Dixon, “Sequence classification of the limit order book using recurrent neural
networks,” Journal of Computational Science, vol. 24, pp. 277–286, 2018. (page 140)

[224] S. Gu, B. Kelly, and D. Xiu, “Empirical asset pricing via machine learning,” The Review
of Financial Studies, vol. 33, no. 5, pp. 2223–2273, 2020. (page 140)

[225] K. Takahashi, K. Yamamoto, A. Kuchiba, and T. Koyama, “Confidence interval for

micro-averaged F 1 and macro-averaged F 1 scores,” Applied Intelligence, vol. 52,
pp. 4961–4972, mar 2022. (page 141)

[226] G. Santafe, I. Inza, and J. A. Lozano, “Dealing with the evaluation of supervised
classification algorithms,” Artificial Intelligence Review, vol. 44, pp. 467–508, jun 2015.
(page 141)

[227] C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval.

Cambridge University Press, 2008. (page 141)

[228] W. Mander, “On the consistency of pantheism,” European Journal for Philosophy of
Religion, vol. 9, no. 1, pp. 1–17, 2017. (page 141)

[229] C.-J. L. Chih-Wei Hsu, Chih-Chung Chang, “A practical guide to support vector classifi-
cation,” Technical Report 1, National Taiwan University, Taipei, 2008. (page 141)

[230] K. Bechler and M. Ludkovski, “Order Flows and Limit Order Book Resiliency on the
Meso-Scale,” Market Microstructure and Liquidity, vol. 03, no. 03n04, p. 1850006, 2017.
(page 150)

[231] Z. Czechowski, M. Lovallo, and L. Telesca, “Multifractal analysis of visibility graph-based

Ito-related connectivity time series,” Chaos, vol. 26, p. 23118, feb 2016. (page 150)

[232] A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola, “A kernel

two-sample test,” Journal of Machine Learning Research, 2012. (pages 150 and 153)
264 Bibliography

[233] A. Gretton, R. Herbrich, A. Smola, O. Bousquet, and B. Schölkopf, “Kernel methods for
measuring independence,” Journal of Machine Learning Research, vol. 6, pp. 2075–
2129, 2005. (page 150)

[234] X. Ge and A. Lin, “Multiscale multifractal detrended partial cross-correlation analysis

of Chinese and American stock markets,” Chaos, Solitons and Fractals, vol. 145, apr
2021. (page 151)

[235] Q. Ruan, M. Zhou, L. Yin, and D. Lv, “Hedging effectiveness of Chinese Treasury bond
futures: New evidence based on nonlinear analysis,” Physica A: Statistical Mechanics
and its Applications, vol. 565, mar 2021. (page 151)

[236] L. Zunino, A. Figliola, B. M. Tabak, D. G. Pérez, M. Garavaglia, and O. A. Rosso,

“Multifractal structure in latin-american market indices,” Chaos, solitons and fractals,
vol. 41, no. 5, pp. 2331–2340, 2009. (page 151)

[237] J. R. Thompson and J. R. Wilson, “Multifractal detrended fluctuation analysis: Practical

applications to financial time series,” Mathematics and computers in simulation, vol. 126,
pp. 63–88, 2016. (pages 151 and 156)

[238] C. Aloui, L. Al-Kayed, A. Asadov, and N. Danila, “Geopolitical Risk and Stock-Bond
Interplay: A Comparative Study of Islamic and Conventional Assets in the GCC,”
Defence and Peace Economics, 2023. (page 151)

[239] J. He, X. Gong, and L. Huang, “Wavelet-Temporal Neural Network for Multivariate
Time Series Prediction,” in Proceedings of the International Joint Conference on
Neural Networks, vol. 2021 of IEEE International Joint Conference on Neural Networks
(IJCNN), (New York, USA), Int Neural Network Soc;, IEEE, July 2021. (page 151)

[240] H. Geman, “Stochastic Clock and Financial Markets,” in Aspects of Mathematical

Finance, pp. 37–52, 2008. (page 152)

[241] G. Samorodnitsky, “Heavy Tails,” in Springer Series in Operations Research and

Financial Engineering, pp. 133–173, Springer, 2016. (page 153)
Bibliography
265

[242] I. B. Aban and M. M. Meerschaert, “Shifted Hill’s estimator for heavy tails,” Communi-
cations in Statistics Part B: Simulation and Computation, vol. 30, no. 4, pp. 949–962,
2001. (page 153)

[243] T. McElroy and A. Jach, “Tail index estimation in the presence of long-memory dy-
namics,” Computational Statistics and Data Analysis, vol. 56, pp. 266–282, feb 2012.
(page 153)

[244] D. Heath, S. Resnick, and G. Samorodnitsky, “Patterns of buffer overflow in a class of

queues with long memory in the input stream,” Annals of Applied Probability, vol. 7,
pp. 1021–1057, nov 1997. (page 153)

[245] M. Karsai, K. Kaski, A. L. Barabási, and J. Kertész, “Universal features of correlated

bursty behaviour,” Scientific Reports, vol. 2, may 2012.

[246] X. Gabaix, P. Gopikrishnan, V. Plerou, and H. E. Stanley, “A theory of power-law

distributions in financial market fluctuations,” Nature, vol. 423, no. 6937, pp. 267–270,
2003. (page 153)

[247] G. F. Gu, X. Xiong, Y. J. Zhang, W. Chen, W. Zhang, and W. X. Zhou, “Stylized facts of
price gaps in limit order books,” Chaos, Solitons and Fractals, vol. 88, pp. 48–58, jul
2016. (page 153)

[248] J. Lee and J. Jang, “Bursts and heavy tails of execution time in business processes,”
Production Planning and Control, vol. 25, no. 12, pp. 980–989, 2014. (page 153)

[249] H. Sheng, Y. Q. Chen, and T. Qiu, “Heavy-tailed distribution and local long memory in
time series of molecular motion on the cell membrane,” Fluctuation and Noise Letters,
vol. 10, pp. 93–119, mar 2011. (page 153)

[250] W. Jitkrittum, Z. Szabó, K. Chwialkowski, and A. Gretton, “Interpretable distribution

features with maximum testing power,” Advances in Neural Information Processing
Systems, no. Nips, pp. 181–189, 2016. (page 153)
266 Bibliography

[251] F. Corsi, “A simple approximate long-memory model of realized volatility,” Journal of

Financial Econometrics, vol. 7, no. 2, pp. 174–196, 2009. (page 154)

[252] J.-F. Muzy, E. Bacry, R. Baile, and P. Poggi, “Uncovering latent singularities from multi-
fractal scaling laws in mixed asymptotic regime. application to turbulence,” Europhysics
Letters, vol. 82, p. 60007, jun 2008. (page 154)

[253] C. Axioglou and S. Skouras, “Markets change every day: Evidence from the memory
of trade direction,” Journal of Empirical Finance, vol. 18, no. 3, pp. 423–446, 2011.
(pages 154, 172, and 176)

[254] A. C. Patil and S. Rastogi, “Multifractal Analysis of Market Efficiency across Structural
Breaks: Implications for the Adaptive Market Hypothesis,” Journal of Risk and Financial
Management, vol. 13, p. 248, oct 2020. (page 154)

[255] W. Schadner, “On the persistence of market sentiment: A multifractal fluctuation

analysis,” Physica A: Statistical Mechanics and its Applications, vol. 581, nov 2021.
(pages 154 and 168)

[256] F. Laumann, J. v. Kügelgen, and M. Barahona, “Kernel two-sample and independence

tests for nonstationary random processes,” Engineering Proceedings, vol. 5, no. 1,
2021. (page 155)

[257] K. Chwialkowski, D. Sejdinovic, and A. Gretton, “A wild bootstrap for degenerate

kernel tests,” in Advances in Neural Information Processing Systems (Z. Ghahramani,
M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, eds.), vol. 4, (La Jolla,
California), pp. 3608–3616, NeurIPS, Jan. 2014. (page 156)

[258] C. L. Li, W. C. Chang, Y. Cheng, Y. Yang, and B. Póczos, “MMD GAN: Towards
deeper understanding of moment matching network,” in Advances in Neural Information
Processing Systems, vol. 2017-Decem, pp. 2204–2214, 2017. (page 156)

[259] N. Modarresi, M. Mohammadi, and S. Rezakhah, “Modeling periodic high-frequency

Bibliography
267

intra-day data,” in 3rd International Conference on Data Science and Business Analyt-
ics, (New York, USA), pp. 65–69, VSB - Technical University of Ostrava, IEEE, 2019.
(pages 158 and 227)

[260] A. Gretton, K. Fukumizu, Z. Harchaoui, and B. K. Sriperumbudur, “A fast, consistent

kernel two-sample test,” Advances in Neural Information Processing Systems 22 -
Proceedings of the 2009 Conference, pp. 673–681, 2009. (page 158)

[261] H. Hong and M. Yogo, “What does futures market interest tell us about the macroecon-
omy and asset prices?,” Working Paper 16712, National Bureau of Economic Research,
January 2011. (page 162)

[262] R. S. Gürkaynak, B. Sack, and E. Swanson, “Do Actions Speak Louder Than Words?
The Response of Asset Prices to Monetary Policy Actions and Statements,” Interna-
tional Journal of Central Banking, vol. 1, May 2005.

[263] J. B. Taylor, “The financial crisis and the policy responses: An empirical analysis of
what went wrong,” NBER Working Paper Series, pp. 14631–, 2009. (page 162)

[264] W. Schadner, “An idea of risk-neutral momentum and market fear: Risk-neutral mo-
mentum and market fear,” Finance Research Letters, vol. 37, nov 2020. (page 168)

[265] Y. C. Chan, “Price movement effects on the state of the electronic limit-order book,”
Financial Review, vol. 40, pp. 195–221, may 2005. (page 168)

[266] T. Jaisson, “Liquidity and Impact in Fair Markets,” Market Microstructure and Liquidity,
vol. 01, p. 1550010, dec 2015. (page 168)

[267] C. S. Gillespie, “Fitting heavy tailed distributions: The powerlaw package,” Journal of
Statistical Software, vol. 64, pp. 1–16, feb 2015. (page 171)

[268] J. Alstott, E. Bullmore, and D. Plenz, “Powerlaw: A Python package for analysis of
heavy-tailed distributions,” PLoS ONE, vol. 9, jan 2014. (page 171)
268 Bibliography

[269] M. Magris, J. Kim, E. Rasanen, and J. Kanniainen, “Long-range auto-correlations in

limit order book markets: inter- and cross-event analysis,” in IEEE Symposium Series
On Computational Intelligence (SSCI), pp. 485–491, 2017. (page 172)

[270] L. Zhang, “Nonlinear autoregressive model design and optimization based on ANN for
the prediction of chaotic patterns in EEG time series,” in Lecture Notes in Computational
Vision and Biomechanics (J. Tavares, N. Dey, and A. Joshi, eds.), vol. 32, (Switzerland),
pp. 51–60, Springer International, 2020. (page 172)

[271] M. D. Gould and J. Bonart, “Queue Imbalance as a One-Tick-Ahead Price Predictor

in a Limit Order Book,” Market Microstructure and Liquidity, vol. 02, p. 1650006, sep
2016. (page 172)

[272] J. Hasbrouck, “Trades, quotes, inventories, and information,” Journal of Financial

Economics, vol. 22, pp. 229–252, dec 1988. (page 174)

[273] T. Chordia, J. Hu, A. Subrahmanyam, and Q. Tong, “Order Flow Volatility and Equity
Costs of Capital,” Management Science, vol. 65, pp. 1520–1551, apr 2019. (page 174)

[274] Y. Xu and C. Li, “Liquidity of the Chinese agricultural futures market and its impact on
futures price-based on high-frequency data,” Sustainability (Switzerland), vol. 10, dec
2018. (page 174)

[275] K.-A. Richards, G. W. Peters, and W. Dunsmuir, “Heavy-tailed features and dependence
in limit order book volume profiles in futures markets,” International Journal of Financial
Engineering, vol. 02, sep 2015. (page 175)

[276] F. Lillo and J. D. Farmer, “The long memory of the efficient market,” Studies in Nonlinear
Dynamics and Econometrics, vol. 8, no. 3, 2004. (page 175)

[277] F. Corsi, S. Mittnik, C. Pigorsch, and U. Pigorsch, “The Volatility of Realized Volatility,”
Econometric Reviews, vol. 27, no. 1-3, pp. 46–78, 2008. (page 176)
Bibliography
269

[278] T. Shi, Y. Chen, Y. Liu, and G. Wu, “Visible and near-infrared reflectance spectroscopy-
An alternative for monitoring soil contamination by heavy metals,” Journal of Hazardous
Materials, vol. 265, pp. 166–176, jan 2014. (page 176)

[279] S. Drożdż, R. Kowalski, P. Oświȩcimka, R. Rak, and R. Gȩbarowski, “Dynamical variety

of shapes in financial multifractality,” Complexity, vol. 2018, 2018.

[280] M. Watorek, S. Drożdż, P. Oświȩcimka, and M. Stanuszek, “Multifractal cross-

correlations between the world oil and other financial markets in 2012–2017,” Energy
Economics, vol. 81, pp. 874–885, jun 2019.

[281] H. Lu, L. Liu, Y. N. Li, X. M. Zhao, X. Q. Wang, and Z. G. Cao, “TasselNetV3: Explain-
able Plant Counting with Guided Upsampling and Background Suppression,” IEEE
Transactions on Geoscience and Remote Sensing, vol. 60, 2022. (page 176)

[282] X. Chen, X. Peng, R. Duan, and J. Li, “Deep kernel learning method for SAR image
target recognition,” Review of Scientific Instruments, vol. 88, no. 10, 2017. (page 182)

[283] Y. Zhang and S. Liao, “A kernel perspective for the decision boundary of deep neural
networks,” in Proceedings - International Conference on Tools with Artificial Intelligence,
ICTAI (M. Alamaniotis and S. Pan, eds.), vol. 2020, pp. 653–660, IEEE Comp Soc, Nov.
2020. (page 182)

[284] C. Ji and Y. Fu, “Deep kernel regression with finite learnable kernels,” in Proceedings of
the 15th Asian Conference on Machine Learning (B. Yanikoglu and W. Buntine, eds.),
vol. 222 of Proceedings of Machine Learning Research, (Istanbul, Turkey), Softtech;
Koc Univ, PMLR, Nov. 2023. 15th Asian Conference on Machine Learning (ACML).
(page 182)

[285] A. Canatar and C. Pehlevan, “A kernel analysis of feature learning in deep neural
networks,” in 58TH Annual Allerton Conference On Communication, Control, And
Computing, Coordinated Sci Lab; Univ Illinois Urbana Champaign, 2022. (page 182)
270 Bibliography

[286] T. Welchowski and M. Schmid, “Sparse kernel deep stacking networks,” Computa-
tional Statistics, vol. 34, pp. 993–1014, Sept. 2019. Annual Conference on Statistical
Computing. (page 182)

[287] R. Achour, J. Hattab, and B. Selmi, “New fractal dimensions of measures and decom-
positions of singularly continuous measures,” Fuzzy Sets and Systems, vol. 479, MAR
1 2024. (page 183)

[288] Y. Yujun, L. Jianping, and Y. Yimei, “Multiscale multifractal multiproperty analysis of

financial time series based on renyi entropy,” International Journal of Modern Physics
C, vol. 28, Feb. 2017. (page 183)

[289] R. Trendafilov, “A Study of Information Regimes in the Limit Order Book,” 2012.
(page 185)

[290] E. Panayi and G. W. Peters, “Stochastic Simulation Framework for the Limit Order Book
Using Liquidity Motivated Agents,” SSRN Electronic Journal, 2015. (page 186)

[291] K.-A. Richards, Modelling the Dynamics of the Limit Order Book in Financial Markets.
PhD thesis, University of New South Wales, 2018. (page 186)

[292] F. Guilbaud and H. Pham, “Optimal high-frequency trading with limit and market orders,”
Quantitative Finance, vol. 13, no. 1, pp. 79–94, 2013. (page 187)

[293] K. H. Chung, B. F. Van Ness, and R. A. Van Ness, “Limit orders and the bid-ask spread,”
Journal of Financial Economics, vol. 53, pp. 255–287, aug 1999. (page 188)

[294] W. Ben Omrane, Y. Tao, and R. Welch, “Scheduled macro-news effects on a Euro/US
dollar limit order book around the 2008 financial crisis,” Research in International
Business and Finance, vol. 42, pp. 9–30, dec 2017. (page 190)

[295] M. Bartolozzi, “A multi agent model for the limit order book dynamics,” European
Physical Journal B, vol. 78, no. 2, pp. 265–273, 2010. (page 190)
Bibliography
271

[296] T.-H. Li, I. L. Macdonald, and W. Zucchini, Hidden Markov and Other Models for
Discrete-Valued Time Series, vol. 93. CRC Press, 1998. (pages 191 and 201)

[297] O. Cappe, E. Moulines, and T. Rydén, Inference in hidden Markov models, vol. 6 of
Springer Series in Statistics. New York, USA: Springer, 1 ed., 2005.

[298] H. N. Nagaraja, Inference in Hidden Markov Models, vol. 48 of Springer Series in

Statistics. 175 Fifth Avenue New York, New York 10010 USA: Springer, 1 ed., 2006.
(page 191)

[299] J. Bulla, “Application of hidden Markov models and hidden semi-Markov models to
financial time series,” University Library of Munich, Germany, MPRA Paper, no. 7675,
p. 157, 2006. (page 200)

[300] Z. Ghahramani and M. I. Jordan, “Factorial Hidden Markov Models,” Machine Learning,
vol. 29, no. 2-3, pp. 245–273, 1997. (page 200)

[301] S. Fine, Y. Singer, and N. Tishby, “The hierarchical hidden Markov model: Analysis and
applications,” Machine Learning, vol. 32, pp. 41–62, jul 1998. (page 200)

[302] J. Bulla and I. Bulla, “Stylized facts of financial time series and hidden semi-Markov
models,” Computational Statistics & Data Analysis, vol. 51, no. 4, pp. 2192–2209, 2006.
(page 201)

[303] A. Berlinet and C. Thomas-Agnan, Reproducing Kernel Hilbert Spaces in Probability

and Statistics. Kluwer Academic Publishers, 2004. (pages 203, 204, and 205)

[304] I. Steinwart and A. Christmann, Support Vector Machines. Springer Publishing Com-
pany, Incorporated, 1st ed., 2008. (pages 203, 204, and 205)

[305] N. Aronszajn, “Theory of Reproducing Kernels,” in Transactions of the American Math-

ematical Society (H. G. W. Begehr, R. P. Gilbert, and M. W. Wong, eds.), vol. 68 of
International Society For Analysis, Applications and Computation, (Dordrecht, Nether-
lands), p. 337, Int Soc Anal Applicat & Computat, Springer, 1950. (page 205)
272 Bibliography

[306] G. Kimeldorf and G. Wahba, “Some results on Tchebycheffian spline functions,” Journal
of Mathematical Analysis and Applications, vol. 33, no. 1, pp. 82–95, 1971.
(pages 205 and 206)

[307] C. Boyer, A. Chambolle, Y. De Castro, V. Duval, F. De Gournay, and P. Weiss, “On rep-
resenter theorems and convex regularization,” SIAM Journal on Optimization, vol. 29,
no. 2, pp. 1260–1281, 2019. (page 206)

[308] M. Unser, “A Representer Theorem for Deep Neural Networks,” Journal of Machine
Learning Research, vol. 20, 2019. (page 206)

[309] A. Argyriou, C. A. Micchelli, and M. Pontil, “When is there a representer theorem?

Vector versus matrix regularizers,” Journal of Machine Learning Research, vol. 10,
pp. 2507–2529, nov 2009. (page 206)

[310] A. J. Smola and B. Schölkopf, “A tutorial on support vector regression,” Statistics and
Computing, vol. 14, no. 3, pp. 199–222, 2004. (page 207)

[311] T. Hofmann, B. Schölkopf, and a. J. Smola, “A review of kernel methods in machine

learning,” Technical Report, vol. 156, no. 156, pp. 1–59, 2006.

[312] T. Hofmann, B. Schölkopf, and A. J. Smola, “Kernel methods in machine learning,”

Annals of Statistics, vol. 36, no. 3, pp. 1171–1220, 2008. (page 207)

[313] B. K. Sriperumbudur, K. Fukumizu, and G. R. Lanckriet, “Universality, characteristic

kernels and RKHS embedding of measures,” Journal of Machine Learning Research,
vol. 12, pp. 2389–2410, 2011. (page 209)

[314] B. K. Sriperumbudur, A. Gretton, K. Fukumizu, B. Schölkopf, and G. R. G. Lanckriet,

“Hilbert space embeddings and metrics on probability measures,” Journal of Machine
Learning Research, vol. 11, pp. 1517–1561, aug 2010. (page 209)

[315] C. A. Micchelli, Y. Xu, and H. Zhang, “Universal kernels,” Journal of Machine Learning
Research, vol. 7, no. Dec, pp. 2651–2667, 2006. (page 209)
Bibliography
273

[316] M. G. Genton, “Classes of Kernels for Machine Learning: A Statistics Perspective,”

Journal of Machine Learning Research, vol. 2, no. Dec, pp. 299–312, 2001.
(pages 212 and 213)

[317] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, UK: Cambridge

University Press, 2006. (page 220)

[318] A. Smola, A. Gretton, L. Song, and B. Schölkopf, “A hilbert space embedding for
distributions,” in Lecture Notes in Computer Science (including subseries Lecture
Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (T. E. Hutter Marcus
Servedio Rocco A., ed.), vol. 4754 LNAI of Lectures in Artificial Intelligence, (Springer
Berlin Heidelberg), pp. 13–31, ALT, Springer-Verlag, oct 2007. (page 225)

[319] B. Schölkopf, “An Introduction to Support Vector Machines,” in Recent Advances and
Trends in Nonparametric Statistics (M. G. Akritas and D. N. Politis, eds.), pp. 3–17,
2003. (page 225)

[320] J. P. Cunningham and Z. Ghahramani, “Linear dimensionality reduction: Survey, in-

sights, and generalizations,” Journal of Machine Learning Research, vol. 16, pp. 2859–
2900, dec 2015. (page 225)

[321] B. Schölkopf, A. J. Smola, R. C. Williamson, and P. L. Bartlett, “New support vector

algorithms,” Neural Computation, vol. 12, pp. 1207–1245, may 2000. (page 225)

[322] N. Smith and M. Niranjan, “Data-dependent kernels in SVM classification of

speech patterns,” tech. rep., University of Cambridge Engineering Department
[Link] 2000.

[323] N. Oliver, B. Scholkopf, and A. J. Smola, “Natural Regularization in SVMs,” Advances

in Large Margin Classifiers, vol. 2, pp. 51–60, 2000. (page 225)

[324] E. A. Meirom and P. Kisilev, “NuC-MKL: A convex approach to non linear multiple
kernel learning,” 2016. (page 225)
274 Bibliography

[325] F. Bach, “Exploring large feature spaces with hierarchical multiple kernel learning,” in
Proceedings of the 21st International Conference on Neural Information Processing
Systems, NIPS’08, (NY, USA), p. 105–112, Curran Associates Inc., 2008. (page 225)

[326] R. Kannao and P. Guha, “Success based locally weighted Multiple Kernel combination,”
Pattern Recognition, vol. 68, pp. 38–51, aug 2017. (page 226)

[327] R. Ma and Y. Zhang, “Traders’ behavioral coupling and market phase transition,”
Physica A: Statistical Mechanics and its Applications, vol. 486, pp. 618–627, nov 2017.
(page 226)

[328] W. Zhang, R. Zhang, C. Wu, A. T. C. Goh, S. Lacasse, Z. Liu, and H. Liu, “State-of-
the-art review of soft computing applications in underground excavations,” Geoscience
Frontiers, vol. 11, no. 4, pp. 1095–1106, 2020. (page 226)

[329] M. Belkin, D. Hsu, and P. P. Mitra, “Overfitting or perfect fitting? Risk bounds for
classification and regression rules that interpolate,” in Advances in Neural Information
Processing Systems (S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. CesaBi-
anchi, and R. Garnett, eds.), vol. 2018-Decem of Advances in Neural Information
Processing Systems, pp. 2300–2311, 2018. (page 226)

[330] M. Belkin, D. Hsu, S. Ma, and S. Mandal, “Reconciling modern machine-learning prac-
tice and the classical bias–variance trade-off,” Proceedings of the National Academy
of Sciences of the United States of America, vol. 116, pp. 15849–15854, aug 2019.
(page 226)

[331] C. Cortes, M. Mohri, and A. Rostamizadeh, “Generalization bounds for learning kernels,”
in Proceedings of the 27th International Conference on International Conference on
Machine Learning, ICML’10, (Madison, WI, USA), p. 247–254, Omnipress, 2010.
(page 226)

[332] M. J. Wainwright, High-Dimensional Statistics: A Non-Asymptotic Viewpoint. Cam-

Bibliography
275

bridge Series in Statistical and Probabilistic Mathematics., Cambridge University Press,

2019. (page 226)

[333] A. Rudi, L. Carratino, and L. Rosasco, “FALKON: An optimal large scale kernel method,”
2017. (page 226)

[334] V. Shankar, A. Fang, W. Guo, S. Fridovich-Keil, L. Schmidt, J. Ragan-Kelley, and

B. Recht, “Neural kernels without tangents,” 2020. (page 227)

[335] R. Salakhutdinov, Z. Hu, and E. P. Xing, “Deep Kernel Learning,” in Proceedings of

Machine Learning Research (A. Gretton and C. C. Robert, eds.), vol. 51, (Cadiz, Spain),
pp. 370–378, PMLR, 09–11 May 2016. (page 227)

[336] M. Donini and F. Aiolli, “Learning deep kernels in the space of dot product polynomials,”
Machine Learning, vol. 106, pp. 1245–1269, oct 2017. (page 227)

[337] W. Jitkrittum, P. Sangkloy, B. Schölkopf, H. Kanagawa, J. Hays, and A. Gretton, “Infor-

mative features for model comparison,” in Advances in Neural Information Processing
Systems, vol. 2018, pp. 808–819, Dec. 2018. (page 230)

[338] M. Sugiyama, T. Suzuki, Y. Itoh, T. Kanamori, and M. Kimura, “Least-squares two-

sample test,” Neural Networks, vol. 24, no. 7, pp. 735–751, 2011. (page 230)

[339] A. Gretton, “Notes on mean embeddings and covariance operators,” 2015. Course
Notes, University College London. (page 232)