Enhancing AutoML with Algorithm Selection:
A Path to Better Performance
Yinuo Guo1[0009−0004−6867−6944] , Mustafa Mısır∗1[0000−0002−6885−6775] , and
Zhongyi Hu2[0000−0002−1113−0199]
1
Duke Kunshan University, Division of Natural and Applied Sciences, Kunshan, China
[Link]@[Link], [Link]@[Link]
2
Wuhan University, School of Information Management, Wuhan, China
[Link]@[Link]
Abstract. Machine learning algorithms have been widely adopted across a vari-
ety of application domains. However, training models and evaluating their perfor-
mance is often time-consuming and computationally expensive. Furthermore, the
identification of (near-)optimal learning strategies, including design choices and
hyperparameter configurations, remains a significant challenge, as these decisions
substantially influence the quality of learning outcomes. To address these issues,
Automated Machine Learning (AutoML) has been developed with the aim of
minimizing manual intervention and automating the creation of ready-to-use ma-
chine learning pipelines. Despite the widespread adoption of numerous successful
AutoML systems, their performance is observed to vary across different datasets
and learning scenarios. Algorithm selection has been proposed as a solution to
this limitation, enabling the recommendation of algorithms on a per-instance ba-
sis. In this study, algorithm selection is utilized to enhance the capabilities of
existing AutoML frameworks with minimal additional effort. A comprehensive
empirical evaluation is conducted on 39 diverse tasks from the OpenML plat-
form, involving 6 state-of-the-art AutoML methods. The results demonstrate that
the integration of algorithm selection not only amplifies the strengths of current
AutoML systems but also leads to improved performance and robustness. These
findings highlight the practicality and efficacy of algorithm selection as a critical
advancement for the next generation of AutoML technologies.
Keywords: Algorithm Selection · AutoML · Classification · OpenML
1 Introduction
Artificial Intelligence (AI) [1] has been a part of an increasing trend both in academia
and industry. The primary element in this rise is Machine Learning (ML) [2]. ML is
concerned with the algorithmic aspect of AI, aiming at learning without explicit in-
structions while relying on experience in the form of data. The success of ML in a
wide range of applications depends on the specifications of the learning algorithms,
the quality of the relevant data, and its size. Focusing on the algorithms, their behavior
is affected by their internal design choices and the values assigned to their parame-
ters, i.e., hyper-parameters. Despite the ML success stories, many suppressing the ex-
pert level human performance, determining the right setting of the ML algorithms may
2 Y. Guo et al.
be cumbersome. Hyper-parameter Optimization (HPO) [3] targets discovering the best
possible hyper-parameter setting for a particular ML algorithm concerning a certain
learning task. There have been strong algorithmic development efforts to deliver HPO
approaches, lately even with Large Language Models (LLMs) [4]. Beyond HPO, Auto-
mated ML (AutoML) [5,6,7] as an umbrella term, deals with both the design decisions
and hyper-parameters combined thoroughly. AutoML essentially focuses on all the el-
ements of the ML pipelines, with the goal of fully automating the model generation
process while focusing on individual tasks in those pipelines [8]. That being said, mod-
elling the design choices as parameters allows to approach AutoML as a HPO problem.
Additionally, there have been sub-fields such as Architecture Optimization or Neural
Architecture Search (NAS) [9]. NAS has been particularly popular, for delivering task-
specific, (near-)optimal neural network architectures. This job has been critical, espe-
cially in the area of Deep Learning (DL), to build complex and effective architectures.
Meta-learning [10] is closely related to this field, utilizing learning from previous ex-
periences. When applying a specific ML model to a particular task, the configuration
can be determined by analyzing the interplay between the model and the task. This prior
experience is characterized by the meta-data or meta-features. Combined with the meta-
data, the previous knowledge can be transferred to the current study, resulting in signifi-
cant time savings, especially considering the computational demands of AutoML tasks.
Many AutoML pipelines integrate meta-learning in some way. Auto-sklearn 2.0 [11]
as an example, employs meta-features to define similarities between datasets, including
metrics like the number of instances and missing values, and record the performance of
different algorithms trained on the datasets. When the pipeline encounters a new task,
it recommends an algorithm based on the most similar dataset. Though several suc-
cessful AutoML systems have emerged, none perform better than the others on every
instance. This fact underscores the importance of per-instance-based analysis. One way
to automatically predict the best performance on an instance level is to use Algorithm
Selection (AS) [12]. This study accommodates AS on top of AutoML. The idea is to
utilize existing AutoML systems for offering improved AutoML performance. In other
words, this setup adds another layer on AutoML systems, providing profound meta-
information, similar to the Cross-domain AS (CdAS) idea [13]. It takes the features of
datasets and performance data as input to an dataset-specific AutoML choice.
For the case study, an existing system, ALORS [14], which formulates AS as a
recommendation task, is employed. This choice is motivated by ALORS’s prior appli-
cation on OpenML, a collaborative ML platform. OpenML’s significance stems from a
previous study [11], which provided the experimental setup analyzed in this work. That
study involved 39 classification tasks and reported a comparative evaluation of 6 Au-
toML systems. The dataset features were directly obtained from OpenML. The results
demonstrated that AS outperformed each of the 6 standalone AutoML systems.
In the following parts of this paper, Section 2 briefly provides an outlook both
on AutoML and AS. Section 3 discusses the methodology alongside the experimen-
tal setup. The empirical results and the discussions are then provided in Section 4.
Section 5 finalizes this article by a summary and future research plans.
Enhancing AutoML with Algorithm Selection: A Path to Better Performance 3
2 Background
2.1 Automated Machine Learning (AutoML)
Numerous domains benefit from the power of Machine Learning (ML) techniques. De-
spite their successes, even exceeding expert performance, building a strong ML model
is not always that straightforward. At the best case, it will require a trial-and-error phase
to come up with a proper ML setup. To simplify the development cycle, AutoML sys-
tems efficiently build machine learning models even when users lack related knowledge.
AutoML has been broadly conceptualized as a search and optimization problem. The
search space, Θ, includes the set of all possible configurations for any part of the ML
pipeline. The strategies along with their hyper-parameters for data preparation, feature
engineering, model generation, and model estimation constitutes that search space [8].
θ∗ = arg min L(fθ , Dval )
θ∈Θ
where L refers to the criteria assessing the quality of a model fθ or a loss / objective
function, trained on a dataset Dtrain , using a validation dataset Dval . Additionally, vari-
ous constraints can be identified such as computational budget, the maximum number
of configurations to check, or restrictions come from the underlying hardware both for
training and inference. For explaining the existing AutoML approaches, the key char-
acteristics of several prominent frameworks are outlined below.
– Auto-sklearn 2.0 [11] is an enhanced version of Auto-sklearn 1.0, using a bandit-
based successive halving approach to allocate resources to promising configurations.
It leverages meta-learning to analyze dataset characteristics and employs Bayesian
optimization to construct portfolio and ensemble models for each instance.
– Auto-WEKA (AW) [15], in a way an earlier variant of Auto-sklearn 1.0, on WEKA,
which is a Java-based Machine Learning (ML) environment. It aims to solve the
Combined Algorithm Selection and Hyper-parameter Optimization (CASH) problem
using Sequential Model-based Algorithm Configuration (SMAC) and Tree-structured
Parzen Estimator (TPE). The follow-up work [16] extends the framework from clas-
sification to regression and integrates it into the WEKA ecosystem.
– H2O [17] is a distributed AutoML framework that efficiently explores large datasets.
It combines the fast random search and stacked ensembles methods, which are rel-
atively straightforward compared with the other frameworks. H2O offers various
choices for the regression, classification, and clustering algorithms, including Gradi-
ent Boosting, Deep Learning, Random Forest, and Hierarchical Clustering. It shows
competitive results on the OpenML benchmarks.
– TPOT [18] is a tree-based AutoML pipeline using genetic programming (GP). It
focuses on ML pipelines, including classification, feature preprocessing, selection,
and combination operators. Each operator modifies the dataset, enabling the con-
struction of tree-based pipelines with a top-k selection and final classification unit.
The genetic tree structures are optimized via GP using the NSGA-II strategy and an
evaluate-select-crossover-mutate cycle.
Many promising AutoML systems exist, such as LightAutoML [19], AMLBID [20],
and AlphaD3M [21]. Additionally, cloud-based AutoML platforms like Amazon Sage-
Maker, IBM Auto-AI, and Google Auto-ML have gained significant popularity and are
4 Y. Guo et al.
widely applied in critical domains such as natural language processing and computer
vision. It is also worth noting that existing ML libraries and frameworks provide various
AutoML capabilities [22].
2.2 Algorithm Selection
Recognizing that algorithm performance can vary significantly across the problem in-
stances of all the domains, identifying an optimal algorithm for a given problem solv-
ing task is a pivotal pursuit. Particularly, domains such as Boolean Satisfiability (SAT),
Traveling Salesman Problem (TSP), and Hyper-parameter Optimization (HPO) encounter
challenges due to their expensive optimization or search nature. Consequently, Al-
gorithm Selection (AS) has gained significant attention, from finding the (near-)best
solver(s) for any problem solving tasks.
Traditionally, AS has been built in the form of performance prediction models. The
basic idea is to predetermine the expected performance of a set of algorithms for a target
scenario or a problem instance. Following those prediction values, the best algorithm
expectedly is picked to solve that target problem instance. Formally, AS is based on
F̂ : P × A → R, where P represents the problem domain’s instances and A represents
considered algorithms. The outcome selection can be defined in the way of the mapping
Ŝ : P → A, to determine the optimal solution for
S(i) = argminF (pi , a) , pi ∈ P .
a∈A
Notably, all the instances possess identifiable features fˆ : P → Rd , with d rep-
resenting the number of features per instance [23]. Starting from this basic strategy, a
variety of AS designs have been proposed. One of the well-known AS approaches is
SATzilla [24]. It efficiently solves SAT problems, particularly proven by its success in
the relevant SAT competitions. Following a feature extraction process from the SAT
instances, the existing solvers were run to obtain the pre-trained performance data, and
then deploys classification models for predictive tasks. Some AS methods benefit from
clustering to group similar instances, then link the best algorithmic choice for each in-
stance type. ISAC [25] applies clustering strategies based on the feature space, while
SUNNY [26] utilizes the k-nearest neighbor (KNN). This enables performance esti-
mation within the identified clusters, empowering the anticipation of the most fitting
algorithm for new instances. Additionally, combining ranking and regression [27] di-
rectly predicts the rank of algorithms instead of the computation time or other numeric
results, which could be potentially problematic. Extreme algorithm selection [28] con-
centrates on the optimal algorithm within a pervasive candidate space, considering both
the problem and algorithmic feature spaces. Beyond that, there have been efforts to
deliver feature-free AS systems [29,30].
Nevertheless, significant challenges remain. Two key components for AS—feature
and performance data—are often incomplete due to resource constraints, resulting in
sparse training data. The ability to incorporate such sparse data while maintaining com-
parability is crucial. ALORS [14] addresses this by using matrix completion based on
collaborative filtering (CF) to recommend algorithms.
Enhancing AutoML with Algorithm Selection: A Path to Better Performance 5
3 Methodology
We consider 39 benchmarks from the OpenML website, utilizing 6 AutoML systems:
Auto-sklearn 1.0, Auto-sklearn 2.0, Auto-WEKA, TPOT, H2O, and Tuned Random
Forest. As suggested in the previous section, the compelling performance of ALORS
across various fields convinced us to combine ALORS with the AutoML frameworks.
Nevertheless, any other Algorithm Selection (AS) method could have been used as well,
since the focus is to show the effectiveness of AS.
Algorithm 1 ALORS: cold-start functionality
Input
Rank performance matrix M ∈ Rn×m
Instance features representation X ∈ Rn×d
Description x of the current problem instance
Learning latent factor model Φ
Build E = (xi , Ui ) , xi ∈ Rd , Ui ∈ Rk , i ∈ 1 . . . n
Learn the mapping Φ : Rd 7→ Rk from E
Algorithm selection
Compute Ux = Φ(x) as the latent representation
Estimate the performance of j-th algorithm ⟨Ux , Vj ⟩
Return recommendation argmin ⟨Ux , Vj ⟩
j=1...m
The initial dataset features were retrieved via the OpenML API, with up to 107 fea-
tures per dataset. Many datasets contain significant missing values, and only a dozen
features are common to all datasets. To balance missing value tolerance and feature
count, a 10% threshold for missing values was applied, reducing the feature set to 48.
Missing values were imputed using k-Nearest Neighbors (k-NN) with k = 5. Alter-
natively, a subset of 14 features without any missing values across all datasets was
considered for both training and testing.
Concerning the Algorithm Selection (AS) procedure, Algorithm 1 provides the
partial pseudocode of ALORS [14]. Due to the used of Collaborative Filtering (CF),
ALORS allows building AS models with incomplete performance data. However, that
aspect is not a focus area in this work, thus only its cold-start functionality, so its pure
recommendation capability is detailed. Referring to the earlier, traditional AS formal
description, ALORS also takes an algorithm-problem instance performance matrix and
features representing the instances of the target problem as inputs. Unlike the other AS
systems, ALORS performs performance prediction indirectly, after building a mapping
model between two sets of features. The initially selected, either those 48 or 14 fea-
tures are mapped to the latent (hidden) features extracted from the performance data.
To be specific, Singular Value Decomposition (SVD) is applied for Matrix Factorization
(MF). The resulting features characterize the problem instances in the eye of the algo-
rithms rather than solely relying on those initially chosen features. It should be noted
that, SVD additionally outputs features representing the candidate algorithms which are
those 6 AutoML techniques. SVD simply operates as
M = U ΣV ⊤ ≈E UK · VK⊤
6 Y. Guo et al.
where Σ has non-negative values on the first diagonal in a decreasing order. U and V
are corresponding square matrices. Then, a low-rank-k approximation is performed by
zeroing out rows below the kth row in Σ and rewriting U , V into Uk , Vk , representing
the latent features. The latent features are then predicted through the initially provided
descriptive features for each per-instance selection task. Followingly, those features are
used to calculate the rank predictions of the constitutent AutoML systems. Next, those
predictions are converted into a selection decision, with the AutoML method that is
expected to achieve the best performance among the candidates.
4 Computational Results
As mentioned earlier, the experimental setup is adopted from [11], with features directly
retrieved from OpenML. No special adjustments were made to favor Algorithm Selec-
tion (AS). ALORS, applied as AS, uses 10-fold cross-validation (10-CV) and a rank-5
Singular Value Decomposition (SVD) for factorization. The mapping in ALORS is im-
plemented using Random Forest (RF).
AS 2.0 AS 1.0 AW TPOT H2O TunedRF ALORS
AVG 1.85 3.38 6.72 4.97 2.97 5.26 1.78
STD 0.98 1.56 0.56 1.43 1.37 1.48 0.90
MIN 1 1 5 1 1 1 1
MAX 4 6 7 7 6 7 4
Table 1: Comparison of the AutoML systems against ALORS in terms of ranks, when all the
features are used with 5-NN imputation for the missing values, 48 in total
AS 2.0 AS 1.0 AW TPOT H2O TunedRF ALORS
AVG 1.95 3.41 6.72 4.97 3.03 5.26 1.62
STD 1.07 1.55 0.56 1.43 1.36 1.48 0.9
MIN 1 1 5 1 1 1 1
MAX 4 6 7 7 6 7 4
Table 2: Comparison of the AutoML systems against ALORS in terms of ranks, where all the
features without any missing values, 14 in total, are accommodated
Table 1 and 2 show the performance of each AutoML system alongside with ALORS
using two feature sets. The first one is with the aforementioned 48 features after impu-
tation via 5-Nearest Neighbor (NN) while the latter results belong to the case where 14
features are utilized, without a need of imputation. Both results indicate that ALORS
outperforms those 6 AutoML systems when they are used in a standalone manner. For
the first case (Table 1), the average rank difference between ALORS and AS 2.0 which
is the best performing standalone AutoML strategy is rather close, with the ranks of
1.78 and 1.85, respectively. When those 14 features without any missing values are
used (Table 2), ALORS delivers improved performance achieving the average rank of
1.62 while AS 2.0 comes with the average rank of 1.95. It is rather expected to see bet-
ter performance from the second scenario than the first one. The reasons are being that
Enhancing AutoML with Algorithm Selection: A Path to Better Performance 7
the second feature set size is smaller, better aligned with the small instance set, i.e., the
number of datasets, and more reliable since there is no heavy imputation as in the first
configuration. Additionally, having smaller standard deviation in comparison to AS 2.0
is an indicator of robustness.
Since the ALORS selector is applied in this paper, the similarities of instances are
extracted by SVD. Figure 1 visualizes the cluster separation when a low-rank-5 la-
tent representation is benefited. Those 39 benchmark datasets are separated into two
groups, one consisting of multiple small-sized clusters. Going into the details further,
highly similar datasets can be identified, given as follows: (bank-marketing, nomao),
(albert, robert), (miniboone, kddcup09, mfeat-factors), (fashion-mnist, volkert), (higgs,
shuttle), (dilbert, jannis), (adult, connect-4, vehicle).
Although not all those similar groups are resembling on paper, not even from the
same domains, their expectations in terms of the learning tasks are closely comparable.
This outcome can be largely beneficial for identifying cheaper datasets to be used for
AutoML training by only providing a diverse yet a small dataset.
The effectiveness of the instance clusters is supported by the analysis of features.
The significance of the 48 selected features is portrayed in the accompanying Figure 2,
the left one. This significance is quantified by the Gini Index, a metric derived from
RF computations during the feature mapping process in the ALORS system. The figure
highlights the top 6 crucial features:
– PercentageOfBinaryFeatures
– NumberOfNumericFeatures
– NumberOfInstances
– MinorityClassSize
– MinorityClassPercentage
– MinSkewnessOfNumericAtts.
A similar feature importance analysis is also reflected in Figure 3, where those 14
features without imputation are used. This outcome indicates that the derived selection
models share similar characteristics.
Additionally, the dis/-similarities among the tested AutoML systems are evaluated.
Figure 4 shows that these systems cannot be distinctly clustered. This aligns with
the performance results, indicating no one-size-fits-all solution and highlighting the
interconnected nature of instances. Despite challenges in distinguishing the systems,
ALORS captures subtle differences and enhances overall performance.
5 Conclusion
This paper explores the use of per-instance Algorithm Selection (AS) to enhance Au-
tomated Machine Learning (AutoML). AutoML is designed to optimize the setup for
learning tasks by identifying suitable design choices, components, and hyper-parameter
values. However, no single AutoML system is universally superior due to the limita-
tions posed by the No Free Lunch (NFL) theorem. To address this, AS is leveraged
to identify the most appropriate AutoML system for each task. Specifically, ALORS,
an existing AS method, is utilized to match datasets with the most suitable AutoML
approach among six systems, rather than relying on a single method for all datasets.
data in rank by SVD (k = 5)
Fig. 1: Hierarchical clusters of instances using the latent features extracted from the performance
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
kcl
sylvine
jasmine
riccardo
credit-g
cnae-9
amazon
australian
phoneme
segment
blood-transfusion
covertype
kr-vs-kp
bank-marketing
nomao
albert
robert
car
helena
miniboone
kddcup09
mfeat-factors
dionis
fashion-mnist
volkert
higgs
shuttle
christine
dilbert
jannis
numerai28.6
airlines
adult
connect-4
vehicle
fabert
guillermo
apsfailure
[Link]
Y. Guo et al. 8
Enhancing AutoML with Algorithm Selection: A Path to Better Performance 9
Fig. 2: Importance of the OpenML instance features, using Gini Index / Importance concerning
the full features set with imputation through 5-NN. The features represented in blue indicate the
ones contributing to the AS model significantly more than the others.
Despite a limited setup of 39 datasets, it is demonstrated that applying AS across Au-
toML systems leads to both performance improvement and increased robustness.
As this the first step towards delivering AutoML with improved robustness, there
are many more steps to cover concerning follow-up research. To be specific, only 39
datasets are present referring to the original study providing the experimental setup.
Referring to the comment pointed out earlier, the number of datasets is quite small for
building strong machine learning (ML) models, so the AS models. Thus, they will be
extended by utilizing the existing, other AutoML benchmarks, also referring to the ones
in the area of Neural Architecture Search (NAS) and Hyper-parameter Optimization
10 Y. Guo et al.
Fig. 3: Importance of the OpenML instance features, using Gini Index / Importance concerning
the features which have no missing values across the utilized datasets.
(HPO) particularly. Following this extension, more AutoML or HPO methods will be
accommodated, enhancing both the analysis and quality. Next, since AutoML is mainly
falling into the category of expensive optimization, more criteria beyond performance
will be taken into account such as model building cost for each approach. Alongside
with this idea to explore, the setting will be investigated as a multi-objective task.
Acknowledgments
The research results of this article are sponsored by the Kunshan Municipal Government
research funding.
Enhancing AutoML with Algorithm Selection: A Path to Better Performance 11
1.2
1.0
0.8
0.6
0.4
0.2
0.0
TPOT
AW
TunedRF
AS 1.0
H2O
AS 2.0
Fig. 4: Hierarchical clusters of the AutoML methods using the latent features extracted from the
performance data in rank by SVD (k = 5)
References
1. Jordan, M.I.: Artificial intelligence—the revolution hasn’t happened yet. Harvard Data
Science Review 1(1) (2019) 1–9
2. Jordan, M.I., Mitchell, T.M.: Machine learning: Trends, perspectives, and prospects. Science
349(6245) (2015) 255–260
3. Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., Thomas, J., Ullmann,
T., Becker, M., Boulesteix, A.L., et al.: Hyperparameter optimization: Foundations, algo-
rithms, best practices, and open challenges. Wiley Interdisciplinary Reviews: Data Mining
and Knowledge Discovery 13(2) (2023) e1484
4. Liu, S., Gao, C., Li, Y.: Large language model agent for hyper-parameter optimization. arXiv
preprint arXiv:2402.01881 (2024)
5. Karmaker, S.K., Hassan, M.M., Smith, M.J., Xu, L., Zhai, C., Veeramachaneni, K.: Automl
to date and beyond: Challenges and opportunities. ACM Computing Surveys (CSUR) 54(8)
(2021) 1–36
6. Barbudo, R., Ventura, S., Romero, J.R.: Eight years of automl: categorisation, review and
trends. Knowledge and Information Systems 65(12) (2023) 5097–5149
7. Salehin, I., Islam, M.S., Saha, P., Noman, S., Tuni, A., Hasan, M.M., Baten, M.A.: Automl:
A systematic review on automated machine learning with neural architecture search. Journal
of Information and Intelligence 2(1) (2024) 52–81
8. He, X., Zhao, K., Chu, X.: Automl: A survey of the state-of-the-art. Knowledge-Based
Systems 212 (2021) 106622
9. Ren, P., Xiao, Y., Chang, X., Huang, P.Y., Li, Z., Chen, X., Wang, X.: A comprehensive
survey of neural architecture search: Challenges and solutions. ACM Computing Surveys
(CSUR) 54(4) (2021) 1–34
10. Vanschoren, J.: Meta-learning. In: Automated Machine Learning. Springer (2019) 35–61
12 Y. Guo et al.
11. Feurer, M., Eggensperger, K., Falkner, S., Lindauer, M., Hutter, F.: Auto-sklearn 2.0: Hands-
free automl via meta-learning. The Journal of Machine Learning Research 23(1) (2022)
11936–11996
12. Kerschke, P., Hoos, H.H., Neumann, F., Trautmann, H.: Automated algorithm selection:
Survey and perspectives. Evolutionary Computation 27(1) (2019) 3–45
13. Misir, M.: Cross-domain algorithm selection: Algorithm selection across selection hyper-
heuristics. In: IEEE Symposium Series on Computational Intelligence (SSCI). (2022) 22–29
14. Mısır, M., Sebag, M.: ALORS: An algorithm recommender system. Artificial Intelligence
244 (2017) 291–314
15. Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-weka: Combined selection
and hyperparameter optimization of classification algorithms. In: Proceedings of the 19th
ACM SIGKDD international conference on Knowledge discovery and data mining. (2013)
847–855
16. Kotthoff, L., Thornton, C., Hoos, H.H., Hutter, F., Leyton-Brown, K.: Auto-weka: Automatic
model selection and hyperparameter optimization in weka. Automated machine learning:
methods, systems, challenges (2019) 81–95
17. LeDell, E., Poirier, S.: H2o automl: Scalable automatic machine learning. In: Proceedings
of the AutoML Workshop at ICML. Volume 2020. (2020)
18. Olson, R.S., Bartley, N., Urbanowicz, R.J., Moore, J.H.: Evaluation of a tree-based pipeline
optimization tool for automating data science. In: Proceedings of the Genetic and Evolution-
ary Computation Conference (GECCO). (2016) 485–492
19. Vakhrushev, A., Ryzhkov, A., Savchenko, M., Simakov, D., Damdinov, R., Tuzhilin, A.:
Lightautoml: Automl solution for a large financial services ecosystem. arXiv preprint
arXiv:2109.01528 (2021)
20. Garouani, M., Ahmad, A., Bouneffa, M., Hamlich, M., Bourguin, G., Lewandowski, A.:
Using meta-learning for automated algorithms selection and configuration: an experimental
framework for industrial big data. Journal of Big Data 9(1) (2022) 57
21. Drori, I., Krishnamurthy, Y., Rampin, R., Lourenco, R.d.P., Ono, J.P., Cho, K., Silva, C.,
Freire, J.: Alphad3m: Machine learning pipeline synthesis. arXiv:2111.02508 (2021)
22. Jin, H., Chollet, F., Song, Q., Hu, X.: Autokeras: An automl library for deep learning. Journal
of Machine Learning Research 24(6) (2023) 1–6
23. Tornede, A., Gehring, L., Tornede, T., Wever, M., Hüllermeier, E.: Algorithm selection on a
meta level. Machine Learning 112(4) (2023) 1253–1286
24. Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: SATzilla: portfolio-based algorithm selection
for SAT. Journal of Artificial Intelligence Research 32(1) (2008) 565–606
25. Kadioglu, S., Malitsky, Y., Sellmann, M., Tierney, K.: ISAC–instance-specific algorithm
configuration. In: Proceedings of the 19th European Conference on Artificial Intelligence
(ECAI). (2010) 751–756
26. Amadini, R., Gabbrielli, M., Mauro, J.: Sunny: a lazy portfolio approach for constraint
solving. Theory and Practice of Logic Programming 14 (2014) 509–524
27. Hanselle, J., Tornede, A., Wever, M., Hüllermeier, E.: Hybrid ranking and regression for al-
gorithm selection. In: German Conference on Artificial Intelligence (Künstliche Intelligenz),
Springer (2020) 59–72
28. Tornede, A., Wever, M., Hüllermeier, E.: Extreme algorithm selection with dyadic feature
representation. In: International Conference on Discovery Science, Springer (2020) 309–324
29. Loreggia, A., Malitsky, Y., Samulowitz, H., Saraswat, V.A.: Deep learning for algorithm
portfolios. In: Proceedings of the 13th Conference on Artificial Intelligence (AAAI). (2016)
1280–1286
30. Seiler, M.V., Kerschke, P., Trautmann, H.: Deep-ela: Deep exploratory landscape analysis
with self-supervised pretrained transformers for single-and multi-objective continuous opti-
mization problems. arXiv preprint arXiv:2401.01192 (2024)