1

Toward Automated Algorithm Design:

A Survey and Practical Guide to
Meta-Black-Box-Optimization
Zeyuan Ma , Hongshu Guo , Yue-Jiao Gong , Senior Member, IEEE
Jun Zhang , Fellow, IEEE and Kay Chen TAN , Fellow, IEEE
arXiv:2411.00625v2 [[Link]] 16 Nov 2024

Abstract—In this survey, we introduce Meta-Black-Box- Evolutionary Computation (EC), including Evolutionary Al-
Optimization (MetaBBO) as an emerging avenue within the gorithms (EAs) and Swarm Intelligence (SI), is widely recog-
Evolutionary Computation (EC) community, which incorporates nized as an effective gradient-free approach for solving BBO
Meta-learning approaches to assist automated algorithm design.
Despite the success of MetaBBO, the current literature provides problems [11]. Over the past decades, EC methods have been
insufficient summaries of its key aspects and lacks practical extensively applied to various optimization challenges [12]–
guidance for implementation. To bridge this gap, we offer a [16], due to their simplicity and versatility. Though effective
comprehensive review of recent advances in MetaBBO, providing for solving BBO problems, traditional EC is constrained by the
an in-depth examination of its key developments. We begin no-free-lunch theorem [17], which asserts that no optimization
with a unified definition of the MetaBBO paradigm, followed
by a systematic taxonomy of various algorithm design tasks, algorithm can universally outperform others across all problem
including algorithm selection, algorithm configuration, solution types, leading to performance trade-offs depending on the
manipulation, and algorithm generation. Further, we conceptu- problem’s characteristics. In response, various adaptive and
ally summarize different learning methodologies behind current self-adaptive EC variants [18]–[25] have been developed.
MetaBBO works, including reinforcement learning, supervised These variants leverage historical optimization data for hyper-
learning, neuroevolution, and in-context learning with Large
Language Models. A comprehensive evaluation of the latest parameter control or operator/algorithm selection during opti-
representative MetaBBO methods is then carried out, alongside mization, improving general performance. However, they face
an experimental analysis of their optimization performance, several limitations. 1) Limited generalization: these methods
computational efficiency, and generalization ability. Based on the often focus on a specific set of problems, limiting their
evaluation results, we meticulously identify a set of core designs generalization due to customized designs. 2) Labor-intensive:
that enhance the generalization and learning effectiveness of
MetaBBO. Finally, we outline the vision for the field by providing designing adaptive mechanisms requires both deep knowledge
insight into the latest trends and potential future directions. of EC domain and the target optimization problem, making
Relevant literature will be continuously collected and updated it a complex task. 3) Additional parameters: many adaptive
at [Link] mechanisms introduce extra hyper-parameters, which can sig-
Index Terms—Meta-Black-Box-Optimization, Evolutionary nificantly impact performance. 4) Sub-optimal performance:
Computation, Black-Box-Optimization, Learning to Optimize. despite increased efforts, design biases and delays in reactive
adjustments often lead to sub-optimal outcomes.
Given this, a natural question arises: can we automatically
I. I NTRODUCTION design effective BBO algorithms while minimizing the depen-
dence on expert input? A recently emerging research topic,
Optimization techniques have been central to research for known as Meta-Black-Box-Optimization (MetaBBO) [26], has
decades [1], [2], with methods applied across engineering [3], shown possibility of leveraging the generalization strength of
economics [4], and science [5]. The optimization problems Meta-learning [27] to enhance the optimization performance
can be classified into White-Box [6] and Black-Box [7] of BBO algorithms in the minimal expertise cost. MetaBBO
types. White-Box problems, with transparent structures, allow follows a bi-level paradigm: the meta level typically maintains
efficient optimization using gradient-based algorithms like a policy that takes the low-level optimization information as
SGD [8], Adam [9], and BFGS [10]. In contrast, Black- input and then automatically dictates desired algorithm design
Box Optimization (BBO) only provides objective values for for the low-level BBO optimizer. The low-level BBO process
solutions, making the analysis and search of the problem space evaluates the suggested algorithm design and returns a feed-
even more challenging. back signal to the meta-level policy regarding the performance
gain. The meta-objective of MetaBBO is to meta-learn a policy
Zeyuan Ma, Hongshu Guo, and Yue-Jiao Gong are with the School of that maximizes the performance of the low-level BBO process,
Computer Science and Engineering, South China University of Technology,
Guangzhou 510006, China (E-mail: gongyuejiao@[Link]) over a problem distribution. Once the training completes, the
Jun Zhang is with Nankai University, Tianjin, China and Hanyang Univer- learned meta-level policy can be directly applied to address
sity, Seoul, South Korea. (E-mail: junzhang@[Link]). unseen optimization problems, hence reducing the need for
Kay Chen TAN is with Hong Kong Polytechnic University, Hong Kong,
China. (E-mail: [Link]@[Link]). expert knowledge to adapt BBO algorithms.
Corresponding author: Yue-Jiao Gong Numerous valuable ideas have been proposed and dis-
 2

The content structure of this paper

Section 2: Formal Section 3: Categorization of Section 4: Different Meta- Section 5: Section 6: Section 7:
Definition of MetaBBO MetaBBO by Meta Tasks Level Learning Paradigms Empirical Evaluation Key Design Strategies Vision for the Field

MetaBBO for MetaBBO via Generalization Towards

Development in Benchmarks Neural Network Design
Algorithm Selection Reinforcement Learning Task Mixtures

MetaBBO for MetaBBO via Proof-of-Principle Evaluation

State Feature Design Fully End-to-End Autonomy
Algorithm Configuration Neuroevolution by MetaBox

MetaBBO for MetaBBO via Smarter Integration of

Training Distribution Design
Solution Manipulation Supervised Learning Large Languge Models

MetaBBO for MetaBBO via

Meta-Objective Design
Algorithm Generation In-Context Learning

Fig. 1. Roadmap of the content structure, beginning with a concept introduction, followed by a review of existing methods across different taxonomies, a
evaluation of selected methods, and a summary of key design strategies and future vision.

cussed in existing MetaBBO research. From the perspective Meta-level Algorithm Design Task

of algorithm design tasks (meta tasks) that the meta-level Optimization input output Algorithm
State Feature Design
policy can address, those MetaBBO works can be cate-
gorized into four branches: 1) Algorithm Selection, where meta-train

for solving the given problem, a proper BBO algorithm Meta-objective

is selected by the meta-level policy from a pre-collected accumulate

Optimization Problem
optimizer/operator pool. 2) Algorithm Configuration, where Distribution
the hyper-parameters and/or operators of a BBO algorithm optimize

are adjusted by the meta-level policy to adapt for the given Low-level Optimizer
problem. 3) Solution Manipulation, where the meta-level Low-level BBO Process

policy is trained to act as a BBO algorithm to manipulate

and evolve solutions. 4) Algorithm Generation, where each Fig. 2. A conceptual overview of the bi-level learning framework of
MetaBBO, illustrating the interactions between its core components to clarify
algorithmic component and the overall workflow are generated the overall workflow.
by the meta-level policy as a novel BBO algorithm. From
the perspective of learning paradigms adopted for training
the meta-level policy, different learning methods such as rein- guide that provides a comprehensive experimental evaluation
forcement learning (MetaBBO-RL) [28]–[34], auto-regressive of MetaBBO methods and a summary of key design strategies,
supervised learning (MetaBBO-SL) [35]–[40], neuroevolu- falling short in offering in-depth evaluations or actionable
tion (MetaBBO-NE) [41]–[43], and Large Language Mod- insights for implementing MetaBBO methods.
els (LLMs)-based in-context learning (MetaBBO-ICL) [37], To address the gaps in previous surveys, this paper provides
[44]–[47] have been investigated in existing works. From a more comprehensive coverage of the MetaBBO field. Fig. 1
the perspective of low-level BBO process, MetaBBO has offers a roadmap to help readers quickly navigate the overall
been instantiated to various optimization scenarios such as content structure. We first provide a formal definition of
single-objective optimization [31], [32], [34], multi-objective MetaBBO in Section II. Subsequently, we identify four main
optimization, multi-modal optimization [48], large scale global algorithm design tasks in existing MetaBBO works and their
optimization [40], [42], [43], and multi-task optimization [49], working scenarios in Section III. In Section IV, we further
[50]. Such an intricate combination of algorithm design tasks, elaborate four learning paradigms, with easy-to-follow tech-
learning paradigms, and low-level BBO scenarios makes it nical details. Section V provides a proof-of-principle perfor-
challenging for new practitioners to systematically learn, use, mance evaluation on nine representative MetaBBO methods.
and develop MetaBBO methods. Unfortunately, there is still According to the evaluation results, Section VI provides in-
a lack of a comprehensive survey and practical guide to the depth discussion about the key design strategies in MetaBBO.
advancements in MetaBBO. Finally, we outline the vision for the MetaBBO field in
While some related surveys discussed the integration of Section VII. The contributions of this survey are generally
learning systems into EC algorithm designs, they have several summarized as follows:
limitations: 1) Previous surveys [51]–[53] focus on one or two • The first comprehensive survey that sorts out existing
algorithm design tasks, such as algorithm configuration [53] literature on MetaBBO. We provide a clear categorization
and algorithm generation [51], [52]. These surveys therefore of existing MetaBBO works according to four distinct
show short in providing comprehensive review and comparison meta-level tasks, along with a detailed elaboration of four
analysis on all four design tasks. 2) Some surveys [54]– different learning paradigms behind.
[57] focus on a particular learning paradigm - RL [58]. • A proof-of-principle evaluation is conducted to provide
However, in MetaBBO, various learning paradigms can be practical comparison between MetaBBO works, lead-
adopted, each with distinct characteristics. 3) In addition to ing to an in-depth discussion over several key design
reviewing relevant papers, existing surveys lack a practical strategies related to the learning effectiveness, training
 3

efficiency, and generalization. Meta-level Algorithm Design Task

Problem Optimization input Meta-level output Algorithm Design
• In the end of this paper, we mark several interesting and Distribution State Feature Policy (optimizer index)
promising future research directions of MetaBBO, focus- index
ing different aspects such as the generalization potential, extract feature Low-level
the end-to-end workflow, and the integration of LLMs. sample Optimizer Pool
Problem optimize Low-level Low-level
Instance Optimizer i Optimizer 1

…
II. D EFINITION OF M ETA BBO Low-level BBO Process

Meta-Black-Box-Optimization (MetaBBO) [26] is derived

from the Meta-learning paradigm [27]. Meta-learning is appli- Fig. 3. Conceptual workflow of MetaBBO for Algorithm Selection.
cable to the learning of both models and algorithms [38]. For
example, in [59], a novel Recurrent Neural Network (RNN)
model architecture is meta-learned and subsequently used as meta-level policy πθ is parameterized by learnable parameters
a classifier. In contrast, in [60], a RNN model is meta- θ. It receives sti as input and outputs an algorithm design
learned to serve as a gradient descent algorithm for optimizing ωit , which is then adopted by A to optimize fi . A perfor-
other neural networks. The paradigm in [60] quickly becomes mance measurement function perf(·) is used to evaluate the
popular and the following works explore the possibility of performance gain obtained by this algorithm design decision.
such a paradigm in various white-box optimization scenarios R(·) is accumulated performance gain during the low-level
ranging from first-order optimization [60] to combinatorial op- optimization of a problem instance. We approximate the meta-
timization [61]. To make a distinction with other applications objective J(θ) as the average performance gain across a group
of Learning to Learn, this research line is named by Learning of N problem instances sampled from P, over a certain
to Optimize (L2O). MetaBBO draws key inspiration from number T of optimization steps. To summarize, MetaBBO
the L2O, while targeting black-box optimization scenarios. In aims to search for an optimal meta-level policy πθ∗ which
this section, we provide an overview of the abstract work- maximizes the meta-objective J(θ).
flow shared by existing MetaBBO methods, explaining the
motivation of the core components in MetaBBO. MetaBBO
III. C ATEGORIZATION OF M ETA BBO BY M ETA TASKS
operates within a bi-level framework, as depicted in Fig. 2,
and is detailed as follows. We introduce four common meta-level tasks in MetaBBO:
We begin with the low-level BBO process. A key com- Algorithm Selection in Section III-A, Algorithm Configuration
ponent at this level is the low-level optimizer A. A repre- in Section III-B, Solution Manipulation in Section III-C and
sents a flexible concept, capable of being any off-the-shelf Algorithm Generation in Section III-D. Generally speaking,
EC algorithm, its modern variants, an algorithm pool, or a they are organized by the size of design space, from smallest
structure for creating new algorithms (rather than a specific to largest. Algorithm selection deals with a small space,
existing one). Another crucial element is the optimization choosing from a few BBO optimizers, while algorithm gen-
problem distribution P, representing a collection of optimiza- eration explores a vast space, allowing the meta-level policy
tion problem instances to be solved. Although the size of P to create novel BBO optimizers in an open-ended manner.
could theoretically be infinite, facilitating Meta-learning on Table I presents a selection of works categorized by the meta
an infinite problem set is impossible. In practice, we instead tasks, along with their references, publication years, low-level
sample a collection of N instances {f1 , f2 , ..., fN } from P as optimizers, targeted problem types1 , and technical summaries.
the training set. A meta task T aims to automatically dictate
an algorithm design ω ∈ Ω for the low-level optimizer A for
each problem instance in P, where Ω denotes the algorithm A. Algorithm Selection
design space of A. For instance, in a basic DE optimizer [62],
its algorithm configuration (e.g., values of the two hyper- Algorithm Selection (AS) has been discussed for
parameters F and Cr that control the mutation and crossover decades [127]. The goal of AS is to select the most
strength) can be regarded as an algorithm design space Ω. suitable algorithm from the algorithm pool according to
There are various algorithm design spaces, which are discussed the target task. The motivation of AS is that optimization
in detail in Section III. MetaBBO solves the meta task by behaviors and preferred scenarios vary with the algorithms,
learning a meta-level policy πθ for the algorithm decision. resulting in a notable performance difference [128]. Initially,
Formally, for a meta-level algorithm design task T := AS is performed by human experts, who suggest algorithms
{P, A, Ω}, πθ is trained to maximize the meta-objective J(θ): based on their knowledge, which is labor-intensive and
N T
requires extensive expertise. To alleviate this dependency,
1 XX researchers seek to develop more automated approaches.
J(θ) = Ef ∈P [R (A, πθ , f )] ≈ perf(A, ωit , fi )
N i=1 t=1
1 We use SOP, MOOP, COP, CMOP, MMOP, MMOOP, LSOP, LS-MOOP,
ωit = πθ (sti ), sti = sf(A, fi , t) (1)
MILP and CO to denote single-objective optimization, multi-objective opti-
where sf(·) is a state feature extraction function, which cap- mization, constrained optimization, constrained multi-objective optimization,
multi-modal optimization, multi-modal multi-objective optimization, large-
tures the optimization state information from the interplay scale optimization, large-scale multi-objective optimization, mixed integer
between the optimizer A and the problem instance fi . The linear programming and combinatorial optimization, respectively.
 4

TABLE I
R EPRESENTATIVE WORKS IN M ETA BBO, CATEGORIZED BY DIFFERENT ALGORITHM DESIGN TASKS . W E HAVE PROVIDED AN ONLINE PAGE , WHERE
MORE DETAILS OF EXISTING M ETA BBO WORKS ARE INCLUDED .

Low-level Optimization
Algorithm Year Technical Summary
Optimizer Type
Meta-QAP [63] 2008 MMAS CO per-instance algorithm selection by MLP classifier for Quadratic Assignment Problem (QAP)
Meta-TSP [64] 2011 GA CO per-instance algorithm selection by MLP classifier for Travelling Salesman Problem (TSP)
Algorithm Selection

Meta-MOP [65] 2019 MOEA MOOP per-instance algorithm selection by SVM classifier from ten multi-objective optimizers
Meta-VRP [66] 2019 MOEA CO per-instance algorithm selection by MLP classifier from four multi-objective optimizers
AR-BB [67] 2020 EAs, SI SOP per-instance algorithm selection by symbolic problem representation and LSTM autoregressive prediction
ASF-ALLFV [68] 2022 EAs, SI SOP per-instance algorithm selection by adaptive local landscape feature and KNN classifier
AS-LLM [69] 2024 - SOP per-instance algorithm selection leverage embedding layer in LLMs
HHRL-MAR [70] 2024 SI SOP dynamically switch SI optimizers along the optimization process with a Q-table RL agent
R2-RLMOEA [71] 2024 EAs MOOP dynamically switch 5 EA optimizers along the optimization process with an MLP RL agent
RL-DAS [34] 2024 DE SOP dynamically switch 3 DE optimizers along the optimization process with an MLP RL agent
TransOptAS [72] 2024 EAs, SI SOP per-instance algorithm selection by Transformer performance predictor from single-objective optimizers
RLMPSO [73] 2016 PSO SOP dynamically select PSO update rules
QFA [74] 2018 FA SOP tuning two control parameters of Firefly Algorithm (FA).
RL-MOEA/D [75] 2018 MOEA/D MOOP dynamically control the neighborhood size and the mutation operators used in MOEA/D
QL-(S)M-OPSO [76] 2019 PSO SOP,MOOP dynamically control the parameters of PSO update rule
DE-DDQN [28] 2019 DE SOP mutation operator selection in DE
DE-RLFR [77] 2019 DE MMOOP mutation operator selection in DE for multi-modal multi-objective problems
LTO [78] 2020 CMA-ES SOP dynamically configure the mutation step-size in CMA-ES
QLPSO [30] 2020 PSO SOP dynamically control the inter-particle communication topology of PSO
MARLwCMA [79] 2020 DE SOP mix the strength of mutation adaptation and CMA-ES
LRMODE [80] 2020 DE MOOP incorporate landscape analysis to operator selection
RLDE [81] 2021 DE SOP dynamically adjust the scaling factor F in DE
LDE [31] 2021 DE SOP use LSTM to adaptively control F and CR in DE
RLEPSO [82] 2021 PSO SOP dynamically adjust factors in EPSO
qlDE [83] 2021 DE SOP dynamically determine parameter combinations of F and Cr in DE
DE-DQN [33] 2021 DE SOP mutation operator selection
RLEA-SSC [84] 2021 DE MMOP dynamically determine where to search
RL-PSO [85] 2022 PSO SOP dynamically adjust random values in PSO update rule
RLLPSO [86] 2022 PSO LSOP adaptively adjust the number of performance levels in the population.
MADAC [87] 2022 MOEA/D MOOP dynamically adjust all parameters in MOEA/D by an multi-agent system
Algorithm Configuration

RL-CORCO [88] 2022 DE COP operator selection in constrained problems

MOEA/D-DQN [89] 2022 MOEA/D MOOP leverage DQN to select variation operators in MOEA
RL-SHADE [90] 2022 SHADE SOP perform mutation operator selection in SHADE
RL-HPSDE [29] 2022 DE SOP control parameter sampling method and mutation operator selection
NRLPSO [91] 2023 PSO SOP dynamically adjust learning paradigms and acceleration coefficients
Q-LSHADE [92] 2023 DE SOP dynamically control when to use the scheme to reduce the population.
LADE [93] 2023 DE SOP leverage three LSTM models to generate three sampling distributions of key parameters in DE
LES [42] 2023 CMA-ES SOP use self-attention mechanism to adjust the step size in CMA-ES
RLAM [94] 2023 PSO SOP enhance the PSO convergence by using RL to control the coefficients of the PSO
MPSORL [95] 2023 PSO SOP adaptivly select strategy in multi-strategy PSO
RLDMDE [96] 2023 DE SOP adaptively select mutation strategy of each population in multi-population DE
RLMMDE [97] 2023 MOEA MOOP dynamically determine whether to perform reference point adaptation method
MARLABC [98] 2023 ABC SOP dynamically select optimization strategy
CEDE-DRL [99] 2023 DE COP dynamically select suitable parent population
AMODE-DRL [100] 2023 MODE MOOP two RL agents, one for mutation operator selection, one for parameter tuning
RLHDE [101] 2023 DE SOP use Q-learning to select mutation operators in QLSHADE and control the trigger parameters in HLSHADE
GLEET [32] 2024 PSO,DE SOP dynamic hyper-parameters tuning based on exploration-exploitation tradeoff features
RLMODE [102] 2024 DE MOOP dynamically control the key parameters in DE update rule
RLNS [103] 2024 SSA,PSO,EO MMOP dynamically adjust the subpopulation size
ada-smoDE [104] 2024 DE SOP dynamically control the key parameters in DE update rule
PG-DE [105] 2024 DE SOP dynamic operator selection
SA-DQN-DE [106] 2024 DE MMOP dynamically select proper local search operators
RLEMMO [48] 2024 DE MMOP dynamically select DE mutation operators
MRL-MOEA [107] 2024 MOEA MOOP dynamically select crossover operator in MOEA
MSoRL [108] 2024 PSO LSOP automatically estimate the search potential of each particle
UES-CMAES-RL [109] 2024 UES CMAES SOP determine parameters in restart strategy by RL agent
HF [110] 2024 DE SOP,CO dynamically select DE mutation operators by RL agent or manual mechanism
MTDE-L2T [49] 2024 DE MTOP control parameter F in DE
RNN-OI [38] 2017 - SOP use RNN as a BBO algorithm to output solutions iteratively
RNN-Opt [39] 2019 - SOP using RNN as a algorithm to output sample distribution iteratively
LTO-POMDP [41] 2021 - SOP LSTM-based optimizer to output per-dimensional distribution
MELBA [111] 2022 - SOP use Transformer-based model to output sample distribution
LGA [43] 2023 GA SOP use attention mechanism to imitate crossover and mutation in GA
Solution Manipulation

OPRO [112] 2023 - SOP use LLMs as optimizer to output solutions

LMEA [113] 2023 - SOP use LLMs to select parent solutions and perform crossover and mutation to generate offspring solutions
MOEA/D-LLM [114] 2023 MOEA/D MOOP use LLMs as the optimizer in MOEA/D process
ELM [115] 2023 - CO use LLM agent to generate benchmark programs through evolution of existing ones
ToLLM [116] 2023 - SOP prompt LLMs to generate solutions
GLHF [40] 2024 DE SOP use neural network to imitate mutation and crossover in DE
B2Opt [117] 2024 GA SOP use neural network to imitate operators in GA
RIBBO [37] 2024 - SOP use GPT model to output optimization trajectories
EvoLLM [45] 2024 - SOP imitate ES’s optimization behaviour by iteratively prompting LLM
EvoTF [36] 2024 - SOP use Transformer-based network to output ES’s distribution parameter
LEO [118] 2024 - SOP exploitation via LLM instead of crossover and mutation
CCMO-LLM [119] 2024 - CMOP use LLM as the search operator within a classical CMOEA framework
GSF [120] 2022 - CO generate whole BBO algorithm by using RL agent to select operators from fixed algorithmic template
Algorithm Generation

AEL [121] 2023 - CO use LLM to evolve algorithm source code

EoH [46] 2023 - CO use LLM agent to evolve algorithm’s thoughts and source code
SYMBOL [122] 2024 - SOP automatically generate symbolic update rules along optimization process through LSTM
LLaMEA [123] 2024 - SOP use LLM to evolve EA algorithm
LLMOPT [124] 2024 - MOOP use LLM to evolve operators for multi-objective optimizer
LLaMoCo [35] 2024 - SOP instruction-tuning for LLM to generate accurate algorithm code
OptiMUS [47] 2024 - MILP develop multi-agent pipelines for LLM to solve MILP problem as a professional team
LLM-EPS [125] 2024 - - use LLM to generate offspring codes in evolutionary program search
ALDes [126] 2024 - SOP sequentially generate each component in an algorithm through auto-regressive inference
 5

1) Formulation: We examine the common AS paradigm. In and Multi-Layer Perceptron (MLP) for the label prediction. In
the low-level BBO process, the component A = {A1 , ...AK } contrast, to achieve in-depth data mining of the relationship
represents an algorithm pool A with K candidate BBO between the problem structures and the optimizer performance,
algorithms. The algorithm design space Ω = {1, 2..., K} The study in [67] uses symbolic regression techniques to
is the selective space involving all indexes of the candidate recover the mathematical equation of the given problem and
algorithms, where ω ∈ Ω denotes an index of a candidate then leverages a Long Short-Term Memory (LSTM) [131]
from A. For each problem instance fi in the training set, the to auto-regressively predict the desired candidate algorithm.
goal of AS is to output an algorithm decision ωit for fi at each TransOptAS [72] explores the possibility of constructing a
optimization step t. As illustrated in Fig. 3, MetaBBO auto- performance indicator based solely on the raw objective values
mates this task by maintaining a learnable meta-level policy to eliminate the computation cost for computing sf(·). It
πθ with parameters θ, which takes a state feature sti obtained leverages a Transformer [132]-styled architecture that takes
by sf(·) describing the optimization state of this optimization a batch of sampled objective values as input and outputs the
step, and then outputs ωit . The selected candidate algorithm performance of the candidate algorithms through supervision
A[ωit ] is used to optimize fi in the low-level BBO process. Its under the benchmark results. AS-LLM [69] leverages pre-
performance on fi serves as the performance measurement trained LLM embeddings to extract features from the can-
in Eq. (1). MetaBBO aims to find an optimal meta-level didate algorithms and the target optimization problem, then
policy that suggests a best-performing algorithm in A for selects the best algorithm by feature similarities.
each fi at each optimization step t automatically. Suppose Several latest MetaBBO works explored the possibility of
the optimization horizon of the low-level BBO process is T , extending per-instance AS to dynamic AS during the low-
the meta-objective J(θ) of AS is calculated as: level BBO process [34], [70], [71]. Concretely, the meta-level
algorithm design task in this paradigm turns to flexibly suggest
N T
1 XX one candidate algorithm to optimize fi for each optimization
J(θ) ≈ perf(A[ωit ], fi ) (2)
N i=1 t=1 step t. The dynamic AS is regarded as Markov Decision
Process in the mentioned MetaBBO works and hence can be
After training, πθ is expected to select well-matched candidate maximized by using RL to meta-learn an optimal policy. The
algorithms from A for unseen problems. optimization state feature in RL-DAS [34] includes not only
2) Related Works: First, per-instance AS is widely adopted the problem properties but also the dynamic optimization state
in the literature, where a single algorithm is selected for information to support such flexible algorithm switch. The
the entire optimization progress for each specific problem, optimization performance of its learned AS policy is superior
meaning that ωit remains time-invariant. A straightforward to each state-of-the-art individual DE variant in its algorithm
approach to learning an effective meta-level policy for the pool, which demonstrates the effectiveness of using MetaBBO
AS task is to form a logical association between the at- for dynamic AS.
tributes of fi and the algorithm selection decision ωi that 3) Challenges: While past research has made progress in
corresponds to them. Since typically the number of candidate AS, several technical challenges persist:
algorithms in the pool A is finite, many early-stage MetaBBO • The construction of the algorithm pool A requires deep
for AS researches transformed the meta-level learning pro- expertise on the target problem distribution and promising
cess to a classification task [63]–[69], [129], [130]. In their BBO algorithms. A powerful pool should contain diverse
methodologies, the state feature extraction function sf(·) in BBO algorithms to address problems with different char-
Eq. (1) extracts problem characteristics si of fi , which is acteristics. Future work could explore measuring algo-
significant enough to distinguish fi with the other problem rithm diversity and automating pool construction.
instances. A benchmarking process is employed to identify • For per-instance AS, labeling the training set is expensive
the top-performing candidate algorithm for fi . The identified due to the exhaustive search needed to find the optimal
algorithm is then used as the classification label. The meta- algorithm for each instance. Limited candidates and prob-
level policy πθ is regarded as a classifier and hence meta- lem instances lead to generalization issues. In dynamic
trained to achieve maximum prediction accuracy. The state AS, the increased methodological complexity challenges
feature extraction mechanism sf(·) in these works can be the learning effectiveness of RL methods.
very different according to the target optimization problem • The algorithm design space in MetaBBO for AS is
types. Meta-QAP [63], Meta-TSP [64] and Meta-VRP [66] coarse-grained, limited by the performance of individual
construct an information collection termed as meta data for algorithms without tuning their configurations. In the
combinatorial optimization problems, which maintains the next section, we introduce algorithm configuration tasks,
nodes information, edge connections in the graph and con- which offer larger and more fine-grained design spaces.
straints of a problem instance. For continuous single/multi-
objective optimization problem, exploratory landscape analysis
techniques are adopted in [65], [68], [129], [130], which pro- B. Algorithm Configuration
files the objective space characteristics of a problem instance Algorithm configuration (AC) is a key task in optimization,
such as the pareto dominance, convexity, peaks and valleys. since almost all BBO algorithms possess hyper-parameters
These works mainly apply basic classification models such as [133] and optional operators [134] that affect performance. To
Support Vector Machine (SVM), K-Nearest Neighbors (KNN) automate the AC task, various adaptive and self-adaptive BBO
 6

algorithms have been developed in the past decades [135]. Meta-level Algorithm Design Task
Problem Optimization input Meta-level output Algorithm Design
Algorithms like JADE [19] and APSO [20] leverage historical Distribution State Feature Policy (configurations)
optimization data to compute informative decision statistics
such as the potential of the hyper-parameter values and the sample extract feature

Problem optimize Low-level

success rates of the optional operators [136]–[138]. However, Instance Optimizer
as discussed in the introduction, these approaches suffer from Low-level BBO Process
design bias, limited generalization, and high labor costs.
1) Formulation: As shown in Fig. 4, MetaBBO overcomes Fig. 4. Conceptual workflow of MetaBBO for Algorithm Configuration.
the limitations of manual AC techniques by using meta-
learning to develop a meta-level configuration policy. This
policy dynamically adjusts a BBO algorithm throughout the relationships between the hyper-parameters and the resulting
lower-level BBO procedure. More formally: in the low-level exploration-exploitation tradeoff in each specific algorithm.
BBO process, the optimizer A represents the BBO algorithm Since DE is known to be highly sensitive to hyperparameter
to be configured. The algorithm design space Ω is hence the settings, particularly the scaling factor F and the crossover
configuration space of A. The size of Ω can be either infi- probability Cr, many efforts have focused on meta-tuning DE.
nite (with continuous hyper-parameters) or finite (with discrete RLDE [81] propose a simple Q-table policy to adjust F when
hyper-parameters or several optional operators). MetaBBO optimizing the power generation efficiency in solar energy
dictates AC in a dynamic manner: given a problem instance system. It uses a Boolean indicator as the optimization state
fi , at each optimization step t of the low-level BBO process, feature: indicating whether the solution quality is improved
a state feature sti is obtained by sf(·) to describe the state of between two optimization steps. The algorithm design space,
this optimization step. The meta-level policy πθ (sti ) outputs represented as δF ∈ {−0.1, 0, 0.1}, indicates the variation
the algorithm design ωit , which sets the configuration of A as in F for the subsequent optimization step. Following RLDE,
[Link] (ωit ). Then the algorithm is used to optimize fi for the QLDE [83] extends the algorithm design space to five combi-
current optimization step. Suppose the optimization horizon of nations of the parameter values. RLMODE [102] further pro-
the low-level BBO process is T , the meta-objective J(θ) of poses a specific state extraction function for constrained multi-
MetaBBO for AC is formulated as objective optimization, which divided the state feature into
eight possible situations, according to the solution feasibility
N T and the dominance relationship. The algorithm design space is
1 XX
J(θ) ≈ perf([Link](ωit ), fi ) (3) three combinations of different F and Cr settings to represent
N i=1 t=1 different exploration-exploitation tradeoffs. The Q-table agent
is updated by first selecting the most promising combination
MetaBBO for AC improves on human-crafted adaptive and then observing the resulting performance improvement.
methods by meta-learning the configuration policy through For more fine-grained parameter control, LDE [31] firstly
optimizing the meta-objective in Eq. (3), removing the need considers using recurrent neural network (i.e., LSTM) as
for labor-intensive, expert-driven designs. The bi-level meta- the meta-level policy, which extracts hidden state feature for
learning paradigm also enhances generalization, as the policy separate optimization step and outputs the values for F and Cr
can be trained on a large set of problem instances, distilling from a continuous range [0, 1]. The same authors subsequently
configuration strategies that can be applied to new problems. propose LADE [93] as a extension of LDE. Compared to
2) Related Works: In this paper, we further divide existing LDE, LADE aims to control more hyper-parameters including
MetaBBO for AC works into three sub-categories according not only the mutation strength and crossover rate but also
to the configuration space Ω of the low-level BBO algorithm. the update weights. All parameters are represented as matrix
The first sub-category is hyper-parameter optimization (HPO), operations. Instead of using one LSTM for controlling all
where the hyper-parameter values are controlled by the meta- parameters, LADE’s meta-level policy comprises three LSTM
level policy. The second is adaptive operator selection (AOS), networks for controlling these parameters respectively. LADE
where several optional operators is flexibly selected by the show more robust learning effectiveness than LDE. A re-
meta-level policy. The last is the combination of HPO and cent study, L2T [49], employs the MetaBBO framework to
AOS, where Ω is a complex configuration space including both regulate the setting of DE parameters and the likelihood of
hyper-parameters and operators. Next we introduce related knowledge transfer within the multitask optimization working
works in these sub-categories. scenarios. The state feature is represented by the rate of
a) Hyper-parameter Optimization: Several early at- successful transfers and the enhancement in sub-population
tempts meta-learn a configuration policy that dictates a single performance. Despite adapting F and Cr, the control of pop-
hyper-parameter setting throughout the entire process of solv- ulation size is considered in Q-LSHADE [92]. The algorithm
ing a problem instance [139], [140]. Now, most MetaBBO for design space is the decay rate of the linear population size
AC approaches follow the dynamic AC paradigm in Eq. (3), reduction in LSHADE [141], which can take values from
offering a flexible exploration-exploitation tradeoff to further {0, 0.2}. Q-LSHADE also meta-learns a Q-table policy by the
improve the optimization performance. Since different BBO feedback indicating the performance improvement. There are
algorithms have distinct hyper-parameters, existing MetaBBO also several MetaBBO works which facilitate hyper-parameter
for AC works customize their methods to explore the intricate optimization on other algorithms, such as PSO [30], [32],
 7

[76], [82], [85], [86], [94], [103], ES [42], [78], and the policies. DEDQN indicates that the features in DE-DDQN
Firefly algorithm [74]. In addition, a recent work GLEET [32] show certain redundancy and might fall short in capturing the
proposes a general learning paradigm which show generic local landscape features. To address this, DEDQN proposes
HPO ability for both DE and PSO. Due to the space limitation, a feature extraction mechanism inspired from classical fitness
other related works are summarized in Table I. landscape analysis [142]. By using random walk sampling,
b) Adaptive Operator Selection: The works in this line DEDQN computes ruggedness and fitness distance correla-
aims to dynamically switch the operators of the low-level BBO tions in the local landscape. Results show that landscape
algorithms during the optimization process. The majority of features are effective for MetaBBO methods to generalize
them still focus on DE algorithms [28], [33], [79], [80], [88]– across problem types. For addressing multi-objective optimiza-
[90], [96], [101], due to their strong performance and the tion problem, MOEA/D-DQN embeds the information of the
availability of various operators for selection. These works reference vectors in MOEA/D into the state extraction. In par-
share similar methodologies: a mutation operator pool is ticular, for a solution x of which the corresponding reference
maintained, involving representative mutation operators such vector has weights w, the state vector is combining the solution
as DE/rand/2, DE/best/2, DE/current-to-rand/1, DE/current-to- with the weights: {x, w}. The meta-level policy receives this
best/1 and DE/current-to-pbest/1. In order to address different state feature and then suggests an operator combination for the
types of problems, the technical differences in these works next optimization step. There are four operator combinations
revolve around the tailored state feature extraction design and in the algorithm design space of MOEA/D-DQN, involving
the operator pool. The state feature extraction functions in two mutation operators and two crossover operators. To tackle
these works can be divided into two main strategies: discrete multi-modal optimization problem, RLEMMO [48] first clus-
representation and continuous representation. ters solutions to compute the neighbourhood features. The
For discrete state representation, the study in [79] first com- optimization state is then constructed by concatenating the
putes the diversity variation and the performance improvement solution’s optimization progress, population’s distributional
between two consecutive optimization steps as an effective features, and neighbourhood features. RLEMMO designs an
profile of the optimization dynamics. These two indicators, operator pool with five diverse mutation operators, showing
being continuous variables, are then divided into five distinct low-to-high degrees of of exploration-exploitation tradeoff. A
levels each. According to the discretized state feature, a Q- Transformer-like policy is adopted to enhance the information
table policy is constructed to select one operator from an sharing during the population evolution.
operator pool with three candidates. RLHDE [101] uses the c) Hybrid Control: Some MetaBBO works explore other
relative density in the solution space and the objective space AC perspectives [105], [110]. In particular, the combination
against the initial population and objective values to indicate of HPO and AOS has gained significant attention [29], [75],
the convergence trend and the performance improvement. The [87], [91], [100], [101], since learning a meta-level policy
values of the two density indicators are discretized into five in ΩHPO+AOS would probably result in a better AC policy
and four levels respectively, constituting 20 different optimiza- than learning them separately. Nevertheless, this poses a
tion states. The operators pool in RLHDE involves six muta- significant challenge as learning from an expanded algorithm
tion operators, which improve the diversity of the optimiza- design space necessitates more intricate learning strategies and
tion behaviours, hence strengthening the generalization. RL- model frameworks. Thoughtful design is essential to guarantee
CORCO [88] addresses constrained multi-objective optimiza- effective learning.
tion by enhancing the CORCO algorithm through multiple Q- 3) Challenges: Despite their success, existing MetaBBO
table policies. In the algorithm, each sub-population maintains works for AC still face some challenges.
a Q-table, where rows represent nine states indicating different • A certain proportion of existing methods use a very
levels of objective improvement and constraint violation, and limited set of training problems. In particular, some only
columns represent two mutation operators. The policy selects train their meta-level policies on a specific optimization
the appropriate mutation operator to optimize the solution as problem instance, raising doubts about the actual gener-
effectively as possible. alization performance of the resulting policies.
Compared to discrete features, continuous state feature • MetaBBO for AC works operate on the basis of prede-
extraction enables finer state modeling, providing unique rep- fined low-level BBO algorithms. Hence, the performance
resentations for optimization states and leading to smarter de- of these methods is closely tied to the original BBO
cisions by the meta-level policy. For instance, DE-DDQN [28] algorithm. Furthermore, the inherent algorithm structures,
proposes a very comprehensive optimization state extraction optimization logic, and design biases significantly restrict
function, which computes a total of 99 features: the first 19 the algorithm design space. Can we further expand the
features describe the optimization progress and the properties algorithm design space and step out this boundary? In the
of the target optimization problems, while the rest 80 are next two subsections, we introduce two novel categories
statistics describing the optimization potential of the four mu- of MetaBBO works that offer potential solutions.
tation operators in the operator pool. An MLP neural network-
based meta-level policy generates Q-values for the candidate
mutation operators and the one with maximal Q-value is C. Solution Manipulation
chosen for the next optimization step. Following DE-DDQN, So far, we have introduced two basic categories of MetaB-
DEDQN [33] and MOEA/D-DQN [89] also construct MLP BOAC. An intuitive observation is that within the MetaBBO
 8

framework for AS/AC tasks, the low-level BBO procedure Meta-level Algorithm Design Task
necessitates a BBO algorithm as the foundational optimizer, Problem Optimization input Meta-level
Distribution State Feature Policy
which comes with a defined algorithm design space (e.g.,
algorithm pool or configuration space). This leads to two
sample extract feature serve as
limitations. First, it requires expert knowledge to select an
Problem optimize Low-level
appropriate BBO algorithm, otherwise the meta-level policy’s Instance Optimizer
learning effectiveness and overall performance may suffer. Low-level BBO Process
Second, managing both the meta-level policy and the low-level
BBO optimizer simultaneously incurs certain computational Fig. 5. Conceptual workflow of MetaBBO for Solution Manipulation.
costs. To address these limitations, several MetaBBO works
have explored the potential of directly using the meta-level
policy for solution manipulation. This approach integrates X 0 , obtains the corresponding objective values Y 0 , and then
meta-level training and low-level optimization into a single optimizes fi by iteratively inferring the next-step solution. To
entity, eliminating the need for a predefined BBO algorithm. meta-learn a well-performing πθ , the observed improvement
In this framework, the meta-level policy itself functions as per step is computed as the perf(·) function. Once trained, the
an optimization algorithm, directly manipulating candidate LSTM serves as a BBO algorithm and iteratively optimizes
solutions throughout the optimization process. We illustrate the target optimization problem following Eq. (5). Due to
this MetaBBO workflow in Fig. 5, referring to it as MetaBBO the end-to-end inferring process, RNN-OI is shown to be
for solution manipulation (SM). faster in terms of the running time compared to hand-crafted
1) Formulation: To formulate the process of solution ma- algorithms. Following RNN-OI, similar works include RNN-
nipulation in MetaBBO, some clarifications have to be made. Opt [39] improving RNN-OI through input normalization and
First, MetaBBO for SM integrates the functions of meta- constraint-dependent loss function, LTO-POMDP [41] using
level policy and the low-level BBO algorithm into a single neuroevolution to learn the network parameters, MELBA [111]
parameterized agent πθ , removing the need for a traditionally improving the long sequence modelling of RNN/LSTM by
perceived BBO algorithm. Therefore, the meta-level policy introducing Transformer structure, and RIBBO [37] leveraging
πθ , typically a neural network, inherently serves as the BBO efficient and generic behaviour cloning framework to learn an
algorithm. In this case, the algorithm design space Ω turns optimizer that resembles the given teacher optimizer.
to the parameter space of the policy, where each algorithm Nevertheless, the above works still suffer from generaliza-
design ω in this space corresponds to the values of the neural tion limitation and interpretability issues. On the one hand, the
network parameters θ. Given a problem instance fi , at each optimization state features only include the raw population
optimization step t, the optimization state feature sti is first information, which makes the policy easily overfits to the
computed by sf(·). According to sti , the policy (acts as the training problems. On the other hand, the learned policies in
BBO algorithm) πθ optimizes fi for one optimization step, these works shift toward “black-box” systems, which hinders
e.g., reproducing the candidate solutions. The performance further analysis on what they have learned. In the last two
improvement is hence measured as perf(πθ (sti ), fi ). Suppose years, several more interpretable MetaBBO for SM works
the optimization horizon of the low-level BBO process is T , are proposed to address these issues [36], [40], [43], [117].
the meta-objective of MetaBBO for SM is formulated as These works propose using higher-level features as a substitute
N T
for the raw features to achieve generalizable state features
1 XX across diverse problems. Typically, these features include
J(θ) ≈ perf(πθ (sti ), fi ) (4)
N i=1 t=1 the distributional characteristics of the solution space and
the objective space, the rank of objective values, and the
Through maximizing J(θ) over N problem instances in temporal features reflecting the optimization dynamics. They
the training set, a neural network-based BBO algorithm is have proposed several novel architecture designs to make
obtained, functioning similarly to human-crafted BBO algo- the meta-level policy explicitly resembles representative EC
rithms: iteratively optimizes the problem instances. Next, we algorithms such GA [43], [117], DE [40], and ES [36].
next introduce representative MetaBBO works for SM. For instance, LGA [43] designs two attention-based neural
2) Related Works: An intuitive way of resembling the network modules to act as the selection and mutation rate
iterative optimization behaviour by neural networks is con- adaption mechanisms in GA. The parameterized selection
sidering temporal network structure such as recurrent neural module applies cross-attention between the parent population
networks [38], [39], [41], which enable MetaBBO to directly and the child population, and the obtained attention score
adjust candidate solutions over sequential steps. The corre- matrix is used as the selection probability. The parameterized
sponding mathematical formulation is quite straightforward: mutation rate adaption module applies self-attention within
X t , ht = πθ (X t−1 , Y t−1 , ht−1 ), Y t = fi (X t ) (5) the child population, and the obtained attention scores is
used as the mutation rate variation strength. B2Opt [117]
where πθ is an RNN/LSTM, ht is the hidden state. This improves LGA by proposing a novel, fully end-to-end network
paradigm is first adopted in RNN-OI [38], which meta-learns architecture which resembles all algorithmic components in
an LSTM to reproduce candidate solutions. For each fi in the GA, including crossover, mutation, selection. For example, the
training problem set, RNN-OI randomly initializes a solution selection module within B2Opt utilizes a method similar to
 9

the residual connection in Transformer, facilitating the use of Meta-level Algorithm Design Task
matrix operations for selecting populations. By meta-training Problem Optimization input Meta-level
Distribution State Feature Policy
the proposed meta-level policies on the training problem set,
these MetaBBO for SM works show competitive optimization generate
extract feature
performance. In particular, their meta-level policies are trained sample optimize
Problem Low-level
with low dimensional synthetic problems (≤ 10) yet could be Instance Optimizer

directly generalized for solving high dimensional continuous Low-level BBO Process
control problems (> 500), e.g., neuroevolution [143].
With the emergence of LLMs, their ability to understand the Fig. 6. Conceptual workflow of MetaBBO for Algorithm Generation.
reasoning in natural language outlines a novel opportunity for
SM. Related works in this line widely leverage the In-Context D. Algorithm Generation
Learning (ICL) [144] to prompt with general LLMs iteratively
as an analog to BBO algorithms to reproduce solutions. A Besides MetaBBO for SM, an interesting research question
pioneer work is OPRO [112], which first provides LLMs a comes out: whether learning-based systems such as MetaBBO
context of the problem formulation and historical optimization could automatically create (generate) new BBO algorithms
trajectory described in natural language. It then prompts LLMs with competitive optimization performance and minimal ex-
to suggest better solutions based on the provided context. pertise requirement? To this end, MetaBBO for algorithm gen-
This idea soon becomes popular and spreads to multiple eration (AG) presents a different methodology: meta learning
optimization scenarios such as program search [115] combina- a parameterized policy that could discover novel algorithms
torial optimization [113], multi-objective optimization [114], accordingly without the human-expert prior, of which the
[119], large scale optimization problem [45], [118] and prompt workflow is illustrated in Fig. 6. The difference between AG
optimization [44]. The eye-catching advantage of LLM-based and SM is that the meta-level policy in SM plays both the
SM is that it requires minimal expertise - users only need role of the meta-level policy and the low-level optimizer, while
to describe the optimization problem in nature language, and the meta-level policy in AG is trained to output a complete
LLMs handle the rest. optimizer which is used then in the low-level BBO process.
3) Challenges: As a novel direction, MetaBBO for SM 1) Formulation: MetaBBO for AG works construct an
is promising due to the end-to-end manner. However, several algorithm representation space Ω as its design space. For
technical challenges remain: example, Ω can be a algorithm workflow space, a mathematical
expression space or a programming language space, reflect-
• Approaches like RNN-Opt directly learn to manipulate
ing the way humans express algorithms - through modular
candidate solutions without following a specific algorithm
algorithm workflows, symbolic mathematical expressions or
structure. While this provides flexibility, these methods
programming language syntax. For a problem instance fi ,
often lack transparency and clear understanding of their
a concrete algorithm design ωit is output by the meta-level
inner workings. Additionally, due to the complexity of
policy πθ , according to the optimization state feature sti . The
BBO tasks, exploring strong neural networks capable of
sf(·) function, in this case, can incorporate landscape features,
handling diverse, complex problems remains a challenge.
symbolic representations, or natural language descriptions of
• In contrast, methods like LGA closely mimic the structure
fi . The generated ωit can be a complete workflow, a mathe-
and components of existing EAs, making the process
matical expression or a functional program that represents a
more transparent. However, because these approaches
novel BBO algorithm A. the meta-objective of MetaBBO for
resemble existing algorithms, their performance might
AG is to meta learn a policy πθ capable of generating well-
be inherently constrained by the limits of the original
performing algorithms:
methods.
• MetaBBO approaches that use LLMs, while reducing the N T
1 XX
need for manual algorithm design, face significant com- J(θ) ≈ perf(ωit , fi ), ωit = πθ (sti ) (6)
N i=1 t=1
putational overhead. The iterative interactions with LLMs
generate large volumes of tokens, leading to inefficiencies where perf(ωit , fi ) is the one-step optimization performance
in both time and cost. gain of the generated algorithm on fi . Through training the
• Finally, MetaBBO for SM treats the policy itself as the policy across a problem set, the policy is expected to auto-
optimizer, targeting at learning the optimal mapping from matically generate flexible and even novel BBO algorithms
current landscape to next candidate positions. However, to address various optimization problems. Besides, note that
this remains a highly challenging task for continuous MetaBBO for AG could work with varying granularity: a) gen-
BBO tasks. The possible landscapes are diverse and erating a universal algorithm for all problems [46], b) gen-
infinite. As a result, so far, it is very challenging to build erating customized algorithms for each problem [126], and
and train a model that can effectively handle these com- c) generating flexible optimization rules that adapt to each step
plexities in practice. In the next section, we will explore of the optimization process and each specific problem [120],
the “algorithm generation” approach, which leverages the [122]. In Eq. (6), we demonstrate the case c). In contrast, in
meta-level policy as an algorithm discoverer, namely, us- the case a), a single algorithm ω is generated to serve as ωit
ing learning to create new algorithmic workflows, update in Eq. (6). In case b), a problem-specific ωi is generated to
rules, and implementations. serve as ωit for each optimization step in solving fi .
 10

2) Related Works: Creating a comprehensive algorithm crossover operations on code implementations of previous
representation space Ω is crucial for the meta-level policy algorithms. After evolution, the best-so-far algorithm gener-
to produce innovative and efficient BBO algorithms. Current ated shows superior performance to human-crafted heuristics
MetaBBO methodologies for AG can be categorized into three on combinatorial optimization problems. Subsequent works
types based on their formulation of algorithm representation such as LLaMEA [123] and LLMOpt [124] generalize this
space Ω: algorithm workflow composition, mathematical ex- paradigm to continuous BBO scenarios, and LLaMEA is
pressions, or natural/programming languages. shown to be capable of generating a more complex algorithm
First, we introduce the works that perform algorithm that is competitive with CMA-ES. Despite the above works,
workflow composition. GSF [120] first defines an algorithm LLaMoCo [35] offers a novel perspective: instruction-tuning
template for EAs, then uses RL to fill each part of the the general LLMs to act as an expert-level optimization
template with operators from a predefined operator pool. programmer. LLaMoCo allows users to describe their specific
ALDes [126] overcomes the limitation of using fixed-length optimization problems in Python/LaTex formulation, then it
template through autoregresive learning. It first tokenizes the outputs the complete Python implementation of a desired
common algorithmic components and the corresponding con- optimizer for solving the given problems. To achieve this, a
figuration parameters in EAs, as well as the execution work- large-scale benchmarking is conducted to attain thousands of
flows such as loop and condition. Then, the algorithm gener- problem-solver pairs as the expert-level optimization knowl-
ation task turns into a sequence generation task of the tokens. edge. This knowledge is then injected into LLMs through
Concretely, ALDes prepares three types of operators: four instruction tuning. The experimental results in LLaMoCo
“selection for evolution” operators, six evolution operators and demonstrates that a small model (e.g., codeGen-350M) could
five “selection for replacement” operators, each is associated generate superior algorithm program to larger models which
with some hyper-parameters. Given the property of the target are not fine-tuned by LLaMoCo (e.g., GPT-4), underscoring
optimization problem, a Transformer-style policy is used to that domain specific knowledge might be the key for LLMs
auto-regressively select one desired operator and configure its to understand, reasoning and solve optimization problems.
hyper-parameter from the candidate pool of each operator type. 3) Challenges: MetaBBO for AG works operate in a more
The novel workflow generated by ALDes demonstrate superior expressive algorithm design space. The experimental results in
performance compared to several traditional BBO algorithms. some of these works demonstrate that the generated algorithms
Second, we introduce the works that leverage mathe- are on par with or even superior to human-crafted ones.
matical expression to formulate Ω. The motivation behind The generated BBO algorithms can not only be applied to
this line is that the design space of GSF and ALDes is address optimization problems, but also be further analysed by
highly dependent on manual engineering, which may limit human experts for novel insights in developing optimization
the exploration of more novel algorithm structures. SYM- techniques. Nevertheless, there are still several bottlenecks in
BOL [122] addresses this issue by breaking down the up- existing works:
date equations of BBO algorithms into atomic mathematical
• As an early-stage research avenue, related works in this
operators and operands. SYMBOL constructs a token set
area are still limited. More studies are expected to further
of common mathematical symbols used in EAs, such as
unleash the potential of MetaBBO for AG.
{+, −, ×, x, x∗ , x− , x∗i , ∆x, xr , c}. It then designs an LSTM-
• For symbolic system-based generation frameworks such
based policy which is capable of auto-regressively generating
as ALDes and SYMBOL, the token sets are relatively
a sequence of these mathematical symbols. The generated se-
small, which leads to limited representation capability.
quence can be parsed into update equations for optimizing the
How to construct a comprehensive and expressive token
low-level optimization problem. SYMBOL generates flexible
set tailored for BBO algorithm, and how to ensure the
update rules for each optimization step and each problem
learning effectiveness in the enlarged algorithm design
instance, bringing in certain self-adaptation capabilities. Com-
space need further investigation.
parison results show that SYMBOL achieves state-of-the-art
• For LLM-assisted MetaBBO for AG, although the work-
performance among powerful optimizers.
flow promises an efficient development pipeline, the
Third, we introduce works that leverage natural language
computational resources required to obtain a competitive
and programming language to define Ω. All works in this
BBO algorithm are substantial. Besides, these works
line leverage LLMs as their meta-level policies [35], [46],
rely heavily on the prompt engineering, since LLMs are
[47], [121], [123], [124]. The differences lie in the learn-
sensitive to the prompts they receive.
ing methodologies, the generation workflows and the target
problem types. OptiMUS [47] leverages modular-structured
LLM agents to formulate and solve (mixed integer) linear IV. D IFFERENT L EARNING PARADIGMS AT M ETA L EVEL
programming problems. There are four agents in OptiMUS:
formulator, programmer, evaluator, and manager, which con- In this section, we introduce four key learning paradigms
stitute an optimization expert team and automate the algo- behind the majority of existing MetaBBO works: MetaBBO
rithm generation task through their cooperation. To enable with reinforcement learning (MetaBBO-RL), MetaBBO with
more general-purpose algorithm generation, AEL [121] and supervised learning (MetaBBO-SL), MetaBBO with neuroevo-
EoH [46] are inspired by the evolution capability of large lution (MetaBBO-NE) and MetaBBO with in-context learn-
models [115], prompting LLMs to perform mutation and ing (MetaBBO-ICL).
 11

A. MetaBBO-RL features derived from a random walking strategy. The meta-

In MetaBBO-RL, the meta-level algorithm design task is level policy is an MLP Q-agent with three hidden layers. Dur-
modeled as an MDP [58], [145], where the environment is ing the low-level BBO process, the Q-agent outputs Q-values
the low-level BBO process for a given problem instance f . for three candidate DE mutation operators and selects one for
The optimization state feature space for s, the algorithm the current optimization step. The performance improvement
design space Ω, and the performance metric perf(·) serve as after this step serves as a reward. The transition obtained is
the MDP’s state space, action space, and reward function, used to update the Q-agent by Eq. (8). Other works employing
respectively. As a result, the meta-objective defined in Eq. (1) similar methods include R2-RLMOEA [71], DE-DDQN [28],
becomes the expected accumulated reward. While various MADAC [87], MOEA/D-DQN [89], CEDE-DRL [99], SA-
RL techniques can be applied, the choice must be made by DQN-DE [106], UES-CMAES-RL [109], HF [110].
considering the characteristics of the state and action spaces, 3) Continuous State & Continuous Action: Building on
which we categorize into three main types below. the success of RL techniques in continuous control [149],
1) Discrete State & Discrete Action: Tabular Q- some MetaBBO-RL works adopt policy gradient-based meth-
learning [146] and SARSA [58] are value-based RL techniques ods (e.g., REINFORCE [150], A2C [151], PPO [152]) to
that maintain a Q-table to iteratively update state-action values handle both continuous states and algorithm designs. These
based on interactions with the environment. These methods allow for more flexible control of optimization behavior in
have a notable benefit in their straightforward Q-table struc- the low-level BBO process, possibly improving performance.
tures, which facilitates efficient convergence and reliable effec- In this case, a policy neural network πθ is used to output a
tiveness. However, they are confined to MDPs with discrete probability distribution over the algorithm design space based
(finite) state and action spaces. Many MetaBBO-RL works on the optimization state. The gradient ∇θ J(θ) used to update
adopt these methods for their simplicity. In such works, opti- πθ is computed as
mization states and algorithm designs are pre-defined to form T
!
′ ′ ′
the rows and columns of the Q-table. At each optimization step
X
∇θ J(θ) = −∇θ log πθ (ω t | st ) γ t −t perf(st , ω t , f )
t in the low-level BBO process, the meta-level policy suggests t′ =t
an algorithm design ω t according to st and the Q table. Then, (9)
a transition < st , ω t , perf(st , ω t , f ), st+1 > is obtained and We illustrate the method with the representative work
the Q-table is updated as GLEET [32]. In GLEET, the optimization state is represented
by a structured feature set, including low-level information
such as solution/objective space density and performance
Q(st , ω t ) = perf(st , ω t , f ) + γ max Q(st+1 , ω) (7)
ω∈Ω improvement indicators. A Transformer-style policy network
An example of this approach is the QLPSO algorithm [30], (3 layers) outputs the posterior Gaussian distribution for
which dynamically adjusts the particle swarm topology. In each parameter of each individual. The concrete parameter
QLPSO, the optimization states are {L2, L4, L8, L10}, repre- values are then sampled from this distributions for the current
senting different neighborhood size features of particles, with optimization step, and the corresponding reward is assigned.
corresponding actions to either maintain or change the neigh- After completing an optimization episode (T steps), the policy
borhood size. Performance improvements resulting from suc- network πθ is updated by summing the gradients from each
cessful topology adjustments are rewarded. Other works using step, as shown in Eq.(9). Other MetaBBO-RL works employ-
similar methods include RLNS [103], QFA [74], qlDE [83], ing similar methodologies include LTO [78], RLEPSO [82],
RLMPSO [73], DE-RLFR [77], QL-(S)M-OPSO [76], MARL- LDE [31], RL-PSO [85], MELBA [111], MOEADRL [153],
wCMA [79], LRMODE [80], RLEA-SSC [84], RLDE [81], LADE [93], RLAM [94], AMODE-DRL [100], PG-DE [105],
RL-CORCO [88], RL-SHADE [90]. GLEET [32], RLEMMO [48], RL-DAS [34], SYMBOL [122].
2) Continuous State & Discrete Action: While Tabular Q-
learning and SARSA are effective for discrete state spaces,
B. MetaBBO-NE
some MetaBBO scenarios require continuous optimization
states for finer algorithm design. In such cases, the MDP Neuroevolution [154] is a machine learning subfield where
involves an infinite state space, making the Q-table structure neural networks are evolved using EC methods rather than
incompatible. To address this, neural network-based Q-agents, updated by gradient descent. In [7], ES is demonstrated as a
such as DQN [147] and DDQN [148], are employed to handle scalable alternative to RL for MDPs, especially when actions
continuous state features. The Q-agent is updated by mini- have long-lasting effects. This inspired the development of
mizing the estimation error between the target and predicted MetaBBO methods using EC to evolve the polocies, referred to
Q-functions as as MetaBBO-NE. In MetaBBO-NE, the meta-level maintains
1
  a population of policies {πθ1 , . . . , πθK }, with each policy πθk
Loss(θ) = Qθ (st , ω t )− perf(st , ω t , f ) being used to guide the algorithm design task for a training
2
2 (8) problem set. The fitness of each policy is the average perfor-
t+1
+γ max Qθ (s , ω) mance gain across the problem instances in the training set.
ω∈Ω An EC method, such as ES, is employed to iteratively update
A representative example is DEDQN [33], where the opti- the the meta-level policies, and after several generations, the
mization state is represented by four continuous FLA indicator optimal policy πθ∗ is obtained.
 12

Representative works in MetaBBO-NE include LTO- D. MetaBBO-ICL

POMDP [41] and LGA [43]. For example, in LGA, a pop- In-Context Learning (ICL) [144] is a popular paradigm in
ulation of attention-based neural networks is maintained at LLM research, which prompts LLMs with a structured text
the meta-level, where each network functions as a neural GA collection: a task description, several in-context examples, and
to manipulate solutions. The OpenAI-ES [7] is then used a concrete task instruction. This structured prompt enables
to evolve K = 32 such networks over ten 10-dimensional LLMs to reason effectively based on the provided context,
synthetic functions from the COCO benchmark [155]. without requiring gradient descent or parameter updates. In-
stead, LLMs “learn the context” and generate the desired
output. MetaBBO-ICL is closely related to two meta tasks:
C. MetaBBO-SL Solution Manipulation (SM) and Algorithm Generation (AG).
The main distinction between existing works lies in how they
MetaBBO works using the supervised learning paradigm construct effective in-context prompts.
are closely related to the meta task of solution manipula- For the SM task, OPRO [112] introduces optimization via
tion (SM). As we described in Eq. (4), SM aims to learn a iterative prompting. In each iteration, the task description
parameterized meta-level policy πθ as the low-level optimizer. is tailored to the specific problem, including the objective
The optimization process proceeds by iteratively calling πθ function and constraints in natural language. The in-context
to optimize the current (population of) solution(s). A key examples consist of prior optimization trajectories, and the
difference between MetaBBO-SL and MetaBBO-RL is that task instruction asks the LLM to find a solution better than
MetaBBO-SL meta-trains policies using direct gradient de- the previous best. However, this approach faces challenges
scent on an explicit supervising objective. This resembles due to the limited optimization expertise of general LLMs,
regret minimization [156] of the target optimization problem’s which are not typically trained with optimization knowledge
objective function. To illustrate this, let us examine the recent in mind [35]. Recent studies creatively suggest guiding LLMs
work GLHF [40], which proposes an end-to-end MetaBBO to mimic certain EAs [113], which involves directing LLMs
method mimicking a DE algorithm. GLHF unifies the DE to execute mutation, crossover, and elitism strategies on the
mutation and crossover operations as matrix operations and specified in-context examples.
designs πθ as two customized network modules, LMM and For the AG task, a set of optimizer programs is maintained,
LCM, to simulate matrix-based mutation and crossover. The with LLMs used to evolve these programs. A representative
Gumbel-Softmax function is used in the crossover module work in this area is EoH [46], where the task description
to make it differentiable. Given a solution population X t at includes both the optimization problem formulation and a
the optimization step t when optimizing a problem f , the concrete algorithm design task. The LLM is asked to first
πθ in GLHF optimizes X t to generate offspring population: describe a new heuristic and then implement it in Python.
X t+1 = πθ (X t ). The explicit supervising objective in this The in-context examples consist of previously suggested pro-
case is the objective value f (X t+1 ), which serves as a regret grams, while the task instruction provides five evolution
function to minimize. Then the gradient used to update the instructions, each with varying levels of code refinement.
policy at step t is computed as Other related works include AEL [121], LLaMEA [123], and
LLMOPT [124].
∂f (X t+1 ) ∂πθ
∇θ J(θ) ∝ · (10)
∂πθ ∂θ
E. Summary
Minimizing this regret-based objective trains the meta-level In this section, we have introduced four key learning
policy for effective optimization on the target problem. How- paradigms widely adopted in existing MetaBBO works to
ever, the differentiability of f is a requirement, which may meta-train their meta-level policy. Note that although these
not hold for “black-box” problems. In GLHF, synthetic “white- learning paradigms can all be used to develop MetaBBO
box” problems are used for training, and results show that this methods and their applications, there are significant technical
approach generalizes well to black-box tasks. Other works in differences among them. By closely examining the experi-
this line include RNN-OI [38], RNN-Opt [39], B2Opt [117], mental results presented in the original papers and comparing
EvoTF [36], LEO [118], RIBBO [37], NAP [157]. Notably, their specific implementations, we outline several key factors
RIBBO [37] and EvoTF [36] use supervised imitation learning to highlight the characteristics of each learning paradigm. We
to meta-train their policies to mimic a teacher BBO algorithm. present a comparison using six criteria in Table II, include:
For instance, RIBBO uses a GPT architecture to imitate 1) Development difficulty, measured by the complexity of
optimization trajectory from diverse existing BBO algorithms. code implementation. 2) Expertise dependency, indicating how
It tokenizes each of the collected optimization trajectories much degree of expert-level knowledge is required. 3) Data
into a target token sequence {R1 , X 1 , Y 1 , ..., RT , X t , Y T }, utilization efficiency, measured by the theoretical performance
where Rt , X t and Y t are the regret-based explicit super- gain that can be obtained by the same batch of data. 4) Training
vising objective, population positions and objective values efficiency, measured by the training wall time. 5) Inference
respectively. Through behavior cloning, RIBBO trains the efficiency, measured by the wall time required for solving
GPT to mimic these trajectories, resulting in a generalized an optimization problem. 6) Optimization performance, which
optimization behavior. shows the empirical optimization results.
 13

TABLE II
C OMPARISON OF METHODOLOGY CHARACTERISTICS OVER FOUR LEARNING PARADIGMS FOR M ETA BBO.

Development Expertise Data Training Inference Optimization

Publications
difficulty ↓ dependency ↓ utilization ↑ efficiency ↑ efficiency ↑ Performance ↑
Reinforcement Learning ⋆⋆ ⋆⋆ ⋆⋆⋆ ⋆⋆⋆ ⋆⋆⋆ ⋆⋆⋆ [28]–[34], [48], [49], [70], [71] [73]–[111], [120], [122], [126]
Neuroevolution ⋆⋆⋆ ⋆⋆⋆ ⋆ ⋆ ⋆⋆⋆ ⋆⋆ [41]–[43]
Supervised Learning ⋆⋆ ⋆⋆ ⋆⋆ ⋆⋆ ⋆⋆⋆ ⋆⋆⋆ [36]–[40], [63]–[66], [68], [72], [117]
In-Context Learning ⋆ ⋆ ⋆ ⋆ ⋆ ⋆ [35], [45]–[47], [69], [112]–[116] [118], [119], [121], [123]–[125]

To summarize, each learning paradigm has its own advan- JADE [19], GLPSO [21] and CMA-ES [22]. Further, we in-
tages and drawbacks. MetaBBO-ICL, powered by LLMs, re- clude the MetaBBO methods covering all four meta-tasks: RL-
quires the least efforts in development and design. MetaBBO- DAS [34] for algorithm selection; DE-DDQN [28], LDE [31],
RL, despite necessitating specific skills in constructing RL RLEPSO [82], LES [42], and GLEET [32] for algorithm
systems and crafting the MDP for the meta-level task, excels in configuration; SYMBOL [122] for algorithm generation; and
data utilization due to its ability to improve the policy in an RNN-OI [38] and GLHF [40] for solution manipulation. All
unsupervised setting. Besides, many MetaBBO-RL methods baselines are trained for 1.5 × 106 learning steps and tested
apply simple yet powerful tabular Q-learning, which enjoy over 51 independent runs to ensure fairness. Other settings
high training/inferring efficiency with low computational over- follow their original papers.
head. As for the final performance, we empirically observe that 1) Comparison on the Optimization Performance: The
MetaBBO-RL and MetaBBO-SL generally achieve superior AEI score in MetaBox [26] evaluates the overall optimization
optimization results. Concerning the research focus on the four performance of a MetaBBO method by aggregating three key
learning paradigms, we observed that MetaBBO-RL consis- metrics: final optimization results, FEs consumed, and runtime
tently attracts interest from researchers, whereas MetaBBO- complexity, using an exponential average, lager is better.
ICL is rapidly gaining traction with the advent of LLMs.
The left side of Fig. 7 presents the final optimization
accuracy of all baselines on Synthetic BBOB (top) and Real-
V. E MPIRICAL E VALUATION istic Protein Docking (bottom) testsuites, while the right side
A. Development in Benchmarks presents their respective AEI scores. The results show that: 1)
When considering only the final accuracy, MetaBBO methods
For benchmarking BBO optimizers, many well-known test-
such as RL-DAS, LDE, and GLEET achieve comparable or
suites have been extensively studied and developed [158],
even superior performance to traditional BBO optimizers,
[159]. With the ongoing development of BBO, the correspond-
while some other MetaBBO methods still perform inferiorly
ing benchmarks aim to 1) propose more diverse benchmark
compared to traditional BBO methods. This indicates that
problems in synthetic [67], [160]–[165] and realistic [143],
while MetaBBO methods show potential, as an emerging topic,
[166], [167] scenarios; and 2) automate the benchmarking
there is still significant room for improvement. 2) Different
process through a software platform [155], [168]. These
evaluation metrics yield different conclusions regarding per-
traditional BBO benchmarks can serve as evaluation tools
formance. When considering both optimization performance
for MetaBBO methods. However, compared with traditional
and computational overhead, traditional BBO optimizers such
EC algorithms, the system structure of MetaBBO is more
as CMA-ES achieve a significantly better trade-off, as shown
intricate. Its bi-level learning paradigm involves a meta-level
on the right side of Fig. 7. This highlights a potential limita-
policy, a low-level optimizer, the training/testing logic of the
tion of MetaBBO methods: they typically involve additional
entire system, and the interfaces between the meta and lower
computation during the meta-level decision-making process.
levels. This complexity creates a gap between the conventional
3) Comparing the performance on the Synthetic BBOB and
BBO benchmarks and MetaBBO methods. To address this
Protein Docking test suites, we observe that the performance
compatibility issue, a recent work termed MetaBox [26] pro-
gap between MetaBBO methods and traditional BBO opti-
poses the first benchmark platform specifically for developing
mizers narrows on both evaluation metrics as the problem
and evaluating MetaBBO methods. It provides three differ-
type shifts from the relatively simpler synthetic set to the
ent problem collections (Synthetic-10D, Noisy-Synthetic-10D,
more challenging realistic protein docking set. This suggests
Protein-Docking-12D) , along with two different train-test split
that MetaBBO is promising for solving complex optimization
modes (easy and difficult), which benefits MetaBBO’s training
problems. 4) MetaBBO-RL methods (including RL-DAS, DE-
under different problem distributions and difficulties. In the
DDQN, LDE, RLEPSO, SYMBOL) outperform MetaBBO-
next subsection, we provide a proof-of-principle evaluation of
NE methods (LES) and MetaBBO-SL methods (RNN-OI).
several representative MetaBBO methods using MetaBox.
This observation highlights an important future direction for
the MetaBBO domain: analyzing the theoretical performance
B. Proof-of-Principle Evaluation by MetaBox bounds of different MetaBBO methods.
In this section, we use MetaBox [26] to evaluate the per- 2) Comparison on the Learning Capabilities: As a learn-
formance of 3 traditional EC algorithms and 9 representative ing based paradigm, MetaBBO should also be evaluated using
MetaBBO methods. For traditional EC algorithms, we include metrics that reflect its learning effectiveness. Next, we evaluate
 14

Classic Optimizer MetaBBO Classic Optimizer MetaBBO

8
22.61
7.21
7.02 Synthetic-easy Synthetic-easy
7 6.59
6.81
6.53 6.43 Synthetic-difficult Synthetic-difficult
6.24 6.28 6.21
20
Best Objective Value Metric

6.06
6 5.73 5.77 5.81 5.66 5.82
5.54 5.42 5.32
5.34 15.98 16.09
5.11
5 4.80 15 14.29
4.48 13.23 13.40
12.67

AEI
12.06 12.02
4 10.86
9.90 10.22 9.84
10 8.97 9.36
3 7.85 7.53 7.12
2 5.43 5.02
5
0.99 2.92
1 2.16
1.40
0.06 0.02
0 0
1.2 1.6
Protein-easy Protein-easy
Protein-difficult 1.4 Protein-difficult
1.1
1.05 1.04 1.05
Best Objective Value Metric

1.03 1.04 1.19 1.21

1.01 1.01 1.01 1.01 1.01 1.01 1.01 1.01
1.02 1.2 1.11 1.13
0.99 0.99 0.99 1.00 0.99
1.0 0.98 0.97 1.05 1.03 1.02
0.99 0.98 1.01 0.99 1.00 0.98 1.01
0.93 0.94 1.0 0.95 0.93 0.93 0.92 0.91
0.95
0.91
0.90
0.89

AEI
0.9 0.8 0.71 0.70

0.6
0.8
0.4
0.7
0.2

0.6 0.0
DE

SO

ES

SO

ET

DE

SO

ES

SO

ET

F
O

O
DA

LE

DA

LE
BO

BO
LH

LH
LD

LD
Q

Q
N-

N-
A-

A-
LP

EP

LP

EP
LE

LE
JA

JA
DD

DD
G

G
M

M
RN

RN
CM

CM
RL

RL
G

G
RL

RL
G

G
SY

SY
DE

DE
Fig. 7. Performance comparisons. Top Left: Best objective values on synthetic testsuites. Top Right: AEI scores on synthetic testsuites. Bottom Left: Best
objective values on protein docking testsuites. Bottom Right: AEI scores on protein docking testsuites.

RL-DAS DE-DDQN LDE RLEPSO LES GLEET SYMBOL RNN-OI 20

Docking
Protein

- - 0.000% -33.070% 3.989% 0.000% -0.502% -7.006% 0.000% 40.312% 4.824% 0.000% -12.329% -1.846% 0.000% -5.762% 17.355% 0.000% 34.807% 8.060% 0.000% - - 0.000% 15

10

5
Synthetic

-14.766% 0.000% - 2.690% 0.000% -2.564% 2.222% 0.000% -0.017% -0.126% 0.000% -0.023% -18.836% 0.000% 1.802% 3.631% 0.000% 9.091% -7.366% 0.000% 9.091% -4.667% 0.000% -
Noisy Synthetic

10
0.000% 0.901% - 0.000% 9.972% -2.564% 0.000% -7.049% -0.006% 0.000% -10.558% -0.026% 0.000% -5.231% 0.901% 0.000% 20.868% 4.040% 0.000% 2.267% 3.306% 0.000% 6.211% - 15

20
Synthetic Noisy Protein Synthetic Noisy Protein Synthetic Noisy Protein Synthetic Noisy Protein Synthetic Noisy Protein Synthetic Noisy Protein Synthetic Noisy Protein Synthetic Noisy Protein
Synthetic Docking Synthetic Docking Synthetic Docking Synthetic Docking Synthetic Docking Synthetic Docking Synthetic Docking Synthetic Docking

Fig. 8. MGD scores of baselines. The value at i-th row and j-th column is the MGD(i, j), with smaller value indicating better performance.

the MetaBBO methods using the Meta Generalization Decay TABLE III
(MGD) and Meta Transfer Efficiency (MTE) from MetaBox. MTE SCORES OF THE M ETA BBO METHODS IN THE TRANSFER FROM
SOURCE TESTSUITES TO TARGET TESTSUITES .
a) Meta Generalization Decay: MGD measures the
generalization performance of a MetaBBO method for un- Source
Synthetic Noisy-Synthetic Protein-Docking
seen tasks. Concretely, MetaBox trains two models for the Testsuites

MetaBBO method on two source suites (A and B) and test Target Noisy Protein
Synthetic
Protein
Synthetic
Noisy

them on the target suit B. We record the AEI scores of these Testsuites Synthetic Docking Docking Synthetic
RL-DAS fail - 1 - - -
two models on the target suit as AEIA and AEIB respectively.
DE-DDQN -9.5 1 -3.2 fail -1.41 -9.5
The MGD(A, B) is computed as
LDE -0.05 -0.17 0.19 -0.17 1 -0.05
AEIA RLEPSO fail -4.25 -0.39 -4.25 1 fail
MGD(A, B) = 100 × (1 − )%, (11)
AEIB LES -1.65 -0.05 fail -0.05 fail 1
GLEET 1 -0.05 fail -0.05 fail -0.39
A smaller MGD score indicates that the method generalizes SYMBOL -0.16 fail 0.97 -2.17 -0.05 fail
well from A to B. RNN-OI 1 - 0.01 - - -
Fig. 8 shows the MGD plot of MetaBBO baselines, with
GLHF omitted since it does not provide training codes. The
‘-’ indicates that the model fails to generalize to target testsuit. ones (e.g., DE-DDQN with a single MLP) in the generalization
We can observe that: 1) RL-DAS and RNN-OI cannot be evaluation, even though they outperform within the training
generalized from Synthetic-10D to Protein Docking-12D due distribution. Since the generalization performance is closely
to their optimization state features are dimension-dependent, tied to the model capacity and the data scale, further investi-
highlighting the importance of optimization state design. gation into the scaling laws in MetaBBO is highly anticipated.
2) MetaBBO-NE methods (LES) achieves more robust gen-
eralization than MetaBBO-RL (RL-DAS, DE-DDQN, LDE, b) Meta Transfer Efficiency: MTE score measures the
RLEPSO, GLEET, SYMBOL) and MetaBBO-SL (RNN-OI) transfer learning capability. For a MetaBBO method, its MTE
baselines, possible revealing the learning effectiveness advan- from a problem set A to B is computed as:
tage of neuroevolution paradigm due to its global learning
Tfinetune
ability. However, neuroevolution is exponentially resource- MTE(A, B) = 100 × (1 − )%, (12)
consuming as the neuron counts scale, implying a tradeoff Tscratch
between effectiveness and efficiency. 3) Larger models (e.g., where Tscratch is the learning steps used to attain best perfor-
GLEET with 3 Transformer layers) underperform smaller mance when training on B. Tfinetune is the learning steps used
 15

MLP Networks RNN/LSTM Networks Transformer Networks (Sequence Model) Transformer Networks (Population Batch)
Optimization
Status info
Optimization
Status info
... Optimization
Status info
Optimization
Status info
... Optimization Optimization ...
Optimization
Status infos
Optimization
Status infos ...
Status info Status info ... ...
Input Append Append
MLP MLP RNN/ Hidden RNN/ Hidden
LSTM State LSTM State Transformer Transformer Transformer

Output Output
Algorithm
Design
Algorithm
Design
... Algorithm
Design
Algorithm
Design
... Algorithm Algorithm
Design
...
...
Algorithm
...
Algorithm ...
Design Designs Designs
Optimization Optimization Optimization Optimization
t=1 Horizon Horizon Horizon Horizon
t=0 t=0 t=1 t=0 t=1 t=0 t=1

Fig. 9. The workflow of different neural networks used in existing MetaBBO works: MLP, RNN/LSTM and Transformer architectures.

to fine-tune a model trained on A to attain the same perfor- part of Fig. 9). While these approaches improve learning
mance level. A larger MTE score indicates that the knowledge effectiveness by incorporating historical information, training
learned in A can be easily transferred to solve B. Table III on long horizons (often involving hundreds of generations)
presents the MTE scores of all baselines under each pair of using RNN/LSTM can be challenging due to the inherent
source-target problem collections, where ‘fail’ indicates the issues of gradient vanishing or explosion. Subsequent works,
baseline can not be fine-tuned to achieve similar performance such as MELBA [111], RIBBO [37], and EvoTF [36], address
level on the target problems. Results show that: 1) While this limitation by leveraging Transformer architectures for
many baselines highlight their transfer learning ability under better long-sequence modeling. The common workflow in
some cases (e.g., GLEET: Synthetic to Noisy-Synthetic), they these works is illustrated in the third part of Fig. 9, where
show transfer limitations in other cases, suggesting room for a trajectory of historical optimization states is processed by
improvement. 2) The overall transferring performances across the Transformer to inform the next step in algorithm design.
all baselines and problem collections are relatively noisy, 2) Spatial Dependency Architectures: In addition to tem-
making it difficult to determine whether some transfer failures poral properties, a key characteristic of EC is its population-
stem from the algorithm designs or the diversity of the problem based search manner. Recent MetaBBO methods tailor al-
collections. This opens up a research opportunity to explore gorithmic components for each individual in the population,
the relationship between problem diversity and generalization, maximizing flexibility for low-level optimization. As illus-
as well as how to construct “good” training set for MetaBBO. trated in the rightmost part of Fig. 9, works such as LGA
[43], LES [42], B2Opt [117], GLEET [32], RLEMMO [48],
VI. K EY D ESIGN S TRATEGIES and GLHF [40] construct optimization state features as a
According to the evaluation conducted in Section V, we collection of individual optimization states and leverage the
summarize the key design strategies of existing MetaBBO Transformer’s attention mechanism to enhance information
works related to the effectiveness and generalization: neural sharing across the population of individuals. For instance,
network design (Section VI-A), optimization state design GLEET [32] proposes a novel Transformer-style network that
(Section VI-B), training distribution design (Section VI-C), includes a “fully informed encoder” and an “exploration-
and meta-objective design (Section VI-D). exploitation decoder”. The encoder promotes information shar-
ing by applying self-attention to the state features of all
individuals. The decoder then decodes the hyperparameter
A. Neural Network Design values for each individual specifically.
Four common neural network architectures are frequently
adopted: 1) MLP, 2) RNN and LSTM, 3) temporal dependency
Transformer, and 4) spatial dependency Transformer, as illus- B. State Feature Design
trated in Fig. 9 from left to right. The basic MLP (leftmost in A key component for MetaBBO’s generalization across
Fig. 9) is widely used in existing works due to its simplicity diverse optimization problems is the state feature extraction
and efficiency in training and inference. However, the MLP is function sf(·). We identify three types of features: a) Problem
limited in analyzing the temporal and data batch dependencies identification features, which captures the landscape properties
within the low-level BBO process. We next introduce novel of the target problem. b) Population profiling features, which
designs that address these limitations. describes the distribution of solutions in the low-level BBO
1) Temporal Dependency Architectures: The low-level process. c) Optimization progress features, which tracks im-
BBO process involves iterative optimization over T genera- provements in the solution evaluations at each step. Next, we
tions. A basic MLP-based policy may struggle to effectively introduce common practices for preparing these features.
leverage historical information along the optimization trajec- 1) Problem Identification Features: To identify the target
tory. Then, an intuitive solution is to introduce architectures optimization problem, the Exploratory Landscape Analysis
that support temporal sequence modeling. To this end, works (ELA) framework [169] is widely used. ELA includes six
such as RNN-OI [38], RNN-Opt [39], and LTO-POMDP [41] groups of metrics, such as local search, skewness of the
introduce RNNs and LSTMs [131], which integrate historical objective space, and approximated curvature (both first and
optimization information into hidden representations and com- second-order), which provide a comprehensive summary of
bine it with the current optimization state (shown in the second the problem’s landscape properties. To compute ELA features,
 16

2ULJLQDO3UREOHPV 6KLIWHG3UREOHPV $IILQH3UREOHPV

a large number of points are sampled from the BBO problem
  
and used to compute the features. For example, linear and 
 
quadratic models are fitted to the sampled points and their   
objective values, and the resulting model parameters have been   
shown to be useful for differentiating different problems.   
2) Population Profiling Features: In an optimization prob-   
           
lem, the solution population can converge to different regions I I I I I I I

of the fitness landscape. MetaBBO aims to dynamically adapt

algorithm designs to help the low-level optimizer adjust to Fig. 10. The projected 2D ELA distributions of Left: the original BBOB
problems; Middle: the BBOB problems with shifted optimum; Right: the
these diverse regions. To analyze the distribution of the popu- BBOB problems generated by MA-BBOB [165].
lation, Fitness Landscape Analysis (FLA) [142] is commonly
used, providing various indicators, such as Fitness Distance
Correlation [170], Ruggedness of Information Entropy [171], 2) Constructing New Benchmarks: While augmenting
Auto-Correlation Function [172], Dispersion [173], Negative existing standard synthetic functions with shift and rotation
Slope Coefficient [174], and Average Neutral Ratio [175]. transformations improves generalization, there is still room for
Some of these indicators measure local landscape properties greater diversity in the problem set. To illustrate this, we show
based on the population’s location in the fitness space. Pop- the 2D projection of the ELA distribution for some CoCo-
ulation profiling features complement problem identification BBOB problem instances and their transformed counterparts
features, providing the meta-level policy with a more accurate in the left and middle of Fig. 10. The results reveal that the
optimization state for specific optimization steps. transformations introduce some diversity, but the improvement
3) Optimization Progress Features: Optimization progress is still limited. More effective approaches are expected to
features further complement ELA and FLA features by pro- generate novel benchmarks. The recent work MA-BBOB [165]
viding the meta-level policy with additional information on demonstrates that affine combinations of existing synthetic
objective evaluation-related properties, such as the consumed functions can create more diverse instances. This is shown
FEs, and the distribution of the current population along with in the right part of Fig. 10, where the instances generated by
the objective values. These features track the improvement and MA-BBOB scatter over a wider area of the feature space.
convergence of the population. Interestingly, recent MetaBBO
works like DE-DDQN [28], RLEPSO [82], and GLEET [32]
have found that optimization progress features alone can be D. Meta-Objective Design
sufficient for learning a generalizable meta-level policy. A key The meta-objective J(θ) in MetaBBO represents the ex-
reason is that computing ELA/FLA features consumes addi- pected accumulated performance gain perf(·) over the prob-
tional FEs, which reduces the learning steps available for the lems in the training set. In existing MetaBBO works, perf(·) is
meta-level policy, thus degrading both learning effectiveness typically tied to the objective values of the solution population,
and final optimization performance. guiding the meta-level policy toward improved optimization
performance. An intuitive approach is to use an indicator
C. Training Distribution Design function: if performance improves between two optimization
The training problem set is crucial for learning a gener- steps, a positive reward is given; otherwise, a negative or zero
alizable meta-level policy, with diversity being a key factor. reward is assigned. This approach is widely used in early
Early works like RNN-OI [38] were trained on a limited set of MetaBBO works such as DE-DDQN [28], QLPSO [30], and
instances from the CoCo-BBOB test suite. As shown in Fig. 7, MARLwCMA [79].
a narrow training set leads to poor generalization. To enhance 1) Scale Normalization: Nevertheless, this basic approach
the diversity of the training set, two main methodologies are can pose challenges when aiming to precisely assess perfor-
commonly used in existing MetaBBO approaches. mance improvements, which in turn could affect the adapt-
1) Augmenting Existing Benchmarks: Standard BBO ability of the learned policy. An alternative method involves
benchmarks include the CoCo-BBOB [159], [176] and CEC computing perf(·) directly by determining the reduction in the
BBOB-Competition [158], [177] testsuites, which contain ap- objective value, expressed as ∆f t = f ∗,t−1 − f ∗,t . However,
proximately 20–30 synthetic functions with various proper- directly using this absolute objective value descent may lead to
ties like multimodality, non-separability, and non-convexity. unstable learning due to differing objective value scales across
Most MetaBBO works augment these testsuites by mathemat- various optimization problems. To mitigate this issue, recent
ical transformations: given a D-dimensional function instance MetaBBO works apply normalization to the objective descent:
f (x) : RD → R, it can be transformed to a new instance f ∗,t−1 − f ∗,t
f ′ (x) = f (M T (x − o)), where M ∈ RD×D is a rotation perf(·, t) = (13)
f ∗,1 − f ∗
matrix and o ∈ RD is an offset to the optimal. For example,
recent works like GLEET [32] and RL-DAS [34] apply where f ∗,1 denotes the objective value of the best solution
random combinations of shifts and rotations on the CEC2021 in the initialized population, and f ∗ represents the optimum
test suite [158], generating thousands of synthetic instances of f . In practice, f ∗ is unknown because f is a black-box
and significantly improving the generalization performance of function. However, it can be approximated by an efficient BBO
the learned meta-level policy. algorithm running in advance.
 17

2) Sparse Reward Handling: The difficulty of the low- approaches still introduce design elements that rely on expert
level BBO process increases over time. Initially, the objective knowledge to enhance performance. This reliance typically
value may decrease rapidly, but later, the descent slows as con- involves: 1) the low-level optimization state s, often hand-
vergence approaches, often resulting in a sparse reward issue crafted as a feature vector to represent problem properties
in learning systems. This can mislead the learning of the meta- or optimization progress; and 2) the meta-objective, which is
level policy, causing it to favor sub-optimal algorithm designs mostly developer-defined, introducing subjectivity. While ini-
that focus primarily on the early stages of optimization. To tial efforts have been made to automate feature extraction us-
address this, recent works introduce an adaptive performance ing neural networks [180], [181] and employ model-based RL
metric with a scale factor λ(t) to Eq. (13) to amplify the to learn the meta-objective objectively [98], further systematic
performance improvement in the later optimization stages: studies are needed. Besides, MetaBBO focuses on designing
f ∗,t−1 − f ∗,t algorithms in isolation, assuming the optimization problem
perf(·, t) = λ(t) × (14) is predefined and ready for evaluation. In reality, the initial
f ∗,1 − f ∗
step often involves formulating the problem, either through
where the scale factor λ(t) is a incremental function of the manual model construction or data-driven methods. This dis-
optimization step t. For instance, in MADAC [87], λ(t) is connect reveals a major gap in the optimization process. A
2f ∗ −f ∗,t−1 −f ∗,t . However, ablation studies in RL-DAS [34], more integrated approach would involve learning of objective
GLEET [32], RIBBO [37], and GLHF [40] suggest that using formulation, automatic feature extraction, and then customized
the unscaled, exact performance improvement metric without algorithm design. Developing a cohesive pipeline for these
a scaling factor may be more effective. This underscores the steps offers a promising direction for advancing optimization
variability in scaling methods’ effectiveness across different and improving problem-solving in practical applications.
MetaBBO tasks, warranting further investigation.

VII. V ISION FOR THE F IELD C. Smarter Integration of LLMs

A. Generalization Toward Task Mixtures As discussed in previous sections, the emergence of LLMs
has significantly influenced various fields, including optimiza-
A promising direction is the generalization toward a mix-
tion. Over the past two years, there has been rapid growth
ture of tasks. While MetaBBO works have explored various
in research exploring LLM-assisted EC. For those interested
aspects of model generalization, the evaluation and analysis
in these works, please refer to our introduction in Section III
we provide in previous sections outline potential improvement
and Section IV, or the original papers and a recent survey
through advanced learning techniques, e.g., transfer learning
[52]. The diverse paradigms of LLM-assisted EC align well
[178] and multitask learning [179].
with two algorithm design tasks in MetaBBO: using LLMs
First, existing works often focus on algorithm design for
for algorithm generation [46] and solution manipulation [112].
specific optimizers. For instance, methods like LDE [31]
However, the implicit in-context learning used in existing
and GLHF [40] are designed for algorithm configuration or
methods does not involve learning a specific model for the
imitation tasks, but primarily with basic DE. This narrow
algorithm design task. Instead, they rely on prompting a pre-
focus might lead to uncertain performance when applying
trained LLM with carefully engineered contextual content
these methods to other optimizers. A more effective approach
iteratively. The rationality behind is that general LLMs present
would be to create a higher-level framework that defines
powerful conditional reasoning capabilities, allowing them to
MetaBBO tasks across multiple optimizers, establishing a
provide insights, ideas and even code implementations. Yet,
multitask design space. Developing a universal modularization
recent works, such as LLaMoCo [35], have shown that the
paradigm for various optimizers could allow training a meta-
optimization-specific knowledge embedded within LLMs is
level policy that generalizes well across tasks.
often insufficient to provide robust performance in complex
Additionally, existing works focus exclusively on a single
optimization scenarios.
specific problem type. Separate policies are trained for each
This suggests two promising directions: First is the au-
task type, leading to increased complexity. This outlines an
tomated MetaBBO workflow search, leveraging LLMs for
opportunity to develop a unified agent capable of engaging
designing MetaBBO workflow through code generation and
in automatic algorithm design that adapts to various problem
function search. Designing a learning system like MetaBBO
types. This method not only streamlines the optimization
is inherently challenging, as it requires considerable expertise.
process but also aligns more closely with real-world scenarios,
By providing LLMs with foundational principles of MetaBBO,
where practitioners frequently encounter a diverse array of
the chain of thought within the models may uncover novel
problems. To overcome the limitations of existing methods, a
paradigms. Second, enhancing the semantic understanding of
universal problem representation system is essential to bolster
LLMs regrading optimization processes, terminologies, pro-
the generalization across diverse problem domains.
gramming logics, problem descriptions would significantly
elevate their expertise. To achieve this, an interesting direction
B. Fully End-to-End Autonomy is to develop symbolic language tailored to optimization
The main motivation behind MetaBBO is to reduce the domain, establishing a comprehensive grammar system and
labor-intensive need for expert consultation by offering a accumulating sufficient use cases to train a foundation model
general optimization framework. However, existing MetaBBO specifically for optimization.
 18

VIII. C ONCLUSION [11] P. A. Vikhar, “Evolutionary algorithms: A critical review and its future
prospects,” in ICGTSPICC, 2016, pp. 261–265.
In this survey, we provide a comprehensive review of recent [12] J. J. Liang, B. Y. Qu, and P. N. Suganthan, “Problem definitions and
advancements in MetaBBO. As a novel research avenue within evaluation criteria for the CEC 2014 special session and competition
the BBO and EC communities, MetaBBO offers a promising on single objective real-parameter numerical optimization,” Tech. Rep.,
2013.
paradigm for automated algorithm design. Through a bi-level [13] Q. Zhang, A. Zhou, S. Zhao, P. N. Suganthan, W. Liu, S. Tiwari et al.,
data-driven learning framework, MetaBBO is capable of meta- “Multiobjective optimization test instances for the CEC 2009 special
learning effective neural network-based meta-level policies. session and competition,” Tech. Rep., 2008.
[14] S. Das, S. Maity, B.-Y. Qu, and P. N. Suganthan, “Real-parameter
These policies assist with algorithm selection and algorithm evolutionary multimodal optimization—a survey of the state-of-the-
configuration for a given low-level optimizer, as well as to art,” Swarm Evol. Comput., vol. 1, no. 2, pp. 71–88, 2011.
imitate or generate optimizers with certain flexibility. [15] K. Tang, X. Li, P. N. Suganthan, Z. Yang, and T. Weise, “Benchmark
functions for the CEC’2010 special session and competition on large-
Our review begins with the mathematical definition of scale global optimization,” Tech. Rep., 2007.
MetaBBO, clarifying its bi-level control workflow. Next, we [16] Q. Xu, N. Wang, L. Wang, W. Li, and Q. Sun, “Multi-task optimization
systematically explore four main algorithm design tasks where and multi-task evolutionary computation in the past five years: A brief
review,” Mathematics, vol. 9, no. 8, p. 864, 2021.
MetaBBO excels: AS, AC, SM, and AG. Following the [17] D. H. Wolpert and W. G. Macready, “No free lunch theorems for
discussion of these tasks, we examine four methodologies of optimization,” TEC, vol. 1, no. 1, pp. 67–82, 1997.
training MetaBBO: SL, RL, NE, and ICL. We hope these two [18] M. Srinivas and L. M. Patnaik, “Adaptive probabilities of crossover and
mutation in genetic algorithms,” TSMC, vol. 24, no. 4, pp. 656–667,
parts will provide readers with a clear roadmap to quickly 1994.
locate their interested MetaBBO methods. [19] J. Zhang and A. C. Sanderson, “JADE: Adaptive differential evolution
Further, to provide a practical guide in this field, we with optional external archive,” TEC, vol. 13, no. 5, pp. 945–958, 2009.
[20] Z.-H. Zhan, J. Zhang, Y. Li, and H. S.-H. Chung, “Adaptive particle
benchmark representative MetaBBO methods on a customized swarm optimization,” TSMC, vol. 39, no. 6, pp. 1362–1381, 2009.
platform. The results reveal that, while MetaBBO techniques [21] Y.-J. Gong, J.-J. Li, Y. Zhou, Y. Li, H. S.-H. Chung, Y.-H. Shi, and
outperform traditional BBO algorithms in certain optimization J. Zhang, “Genetic learning particle swarm optimization,” TC, vol. 46,
no. 10, pp. 2277–2290, 2015.
scenarios, there remains a significant potential for improve- [22] N. Hansen, “The CMA evolution strategy: A tutorial,” arXiv preprint,
ment in terms of computational complexity, generalization 2016.
capability, and transfer learning efficiency in the current re- [23] R. Tanabe and A. Fukunaga, “Success-history based parameter adap-
tation for differential evolution,” in CEC, 2013, pp. 71–78.
search. Furthermore, we provide in-depth analysis on some [24] R. Tanabe and A. S. Fukunaga, “Improving the search performance
core designs of MetaBBO: the neural network architecture, of shade using linear population size reduction,” in CEC, 2014, pp.
optimization state feature extraction mechanism, training prob- 1658–1665.
[25] V. Stanovov, S. Akhmedova, and E. Semenkin, “Nl-shade-lbc algorithm
lem distribution, and meta-objective design. These insights with linear parameter adaptation bias change for CEC 2022 numerical
offer practical guidelines for researchers and practitioners optimization,” in CEC, 2022, pp. 01–08.
aiming to develop more effective and efficient MetaBBO [26] Z. Ma, H. Guo, J. Chen, Z. Li, G. Peng, Y.-J. Gong, Y. Ma, and Z. Cao,
“Metabox: a benchmark platform for meta-black-box optimization with
methods. At last, we propose several interesting and open- reinforcement learning,” NeurIPS, 2024.
ended future directions for MetaBBO research, encouraging [27] C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for
further exploration and innovation in this promising field. fast adaptation of deep networks,” in ICML, 2017.
[28] M. Sharma, A. Komninos, M. López-Ibáñez, and D. Kazakov, “Deep
reinforcement learning based parameter control in differential evolu-
R EFERENCES tion,” in GECCO, 2019, pp. 709–717.
[29] Z. Tan, Y. Tang, K. Li, H. Huang, and S. Luo, “Differential evolution
[1] Y. Jin and J. Branke, “Evolutionary optimization in uncertain with hybrid parameters and mutation strategies based on reinforcement
environments-a survey,” TEC, vol. 9, no. 3, pp. 303–317, 2005. learning,” Swarm Evol. Comput., vol. 75, p. 101194, 2022.
[2] S. Sun, Z. Cao, H. Zhu, and J. Zhao, “A survey of optimization methods [30] Y. Xu and D. Pi, “A reinforcement learning-based communication
from a machine learning perspective,” TC, vol. 50, no. 8, pp. 3668– topology in particle swarm optimization,” Neural Comput. Appl.,
3681, 2019. vol. 32, pp. 10 007–10 032, 2020.
[3] M. H. Yar, V. Rahmati, and H. R. D. Oskouei, “A survey on evolu- [31] J. Sun, X. Liu, T. Bäck, and Z. Xu, “Learning adaptive differential
tionary computation: Methods and their applications in engineering,” evolution algorithm from optimization experiences by policy gradient,”
Mod. Appl. Sci, vol. 10, no. 11, p. 131139, 2016. TEC, vol. 25, no. 4, pp. 666–680, 2021.
[4] A. Ponsich, A. L. Jaimes, and C. A. C. Coello, “A survey on [32] Z. Ma, J. Chen, H. Guo, Y. Ma, and Y.-J. Gong, “Auto-configuring
multiobjective evolutionary algorithms for the solution of the portfolio exploration-exploitation tradeoff in evolutionary computation via deep
optimization problem and other finance and economics applications,” reinforcement learning,” in GECCO, 2024, pp. 1497–1505.
TEC, vol. 17, no. 3, pp. 321–344, 2012. [33] Z. Tan and K. Li, “Differential evolution with mixed mutation strategy
[5] A. M. Gopakumar, P. V. Balachandran, D. Xue, J. E. Gubernatis, and based on deep reinforcement learning,” Appl. Soft Comput., vol. 111,
T. Lookman, “Multi-objective optimization for materials discovery via p. 107678, 2021.
adaptive design,” Sci. Rep., vol. 8, no. 1, p. 3738, 2018. [34] H. Guo, Y. Ma, Z. Ma, J. Chen, X. Zhang, Z. Cao, J. Zhang, and Y.-J.
[6] G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm Gong, “Deep reinforcement learning for dynamic algorithm selection:
for deep belief nets,” Neural Comput., vol. 18, no. 7, pp. 1527–1554, A proof-of-principle study on differential evolution,” TSMC, vol. 54,
2006. no. 7, pp. 4247–4259, 2024.
[7] T. Salimans, J. Ho, X. Chen, S. Sidor, and I. Sutskever, “Evolution [35] Z. Ma, H. Guo, J. Chen, G. Peng, Z. Cao, Y. Ma, and Y.-J. Gong,
strategies as a scalable alternative to reinforcement learning,” arXiv “LLaMoCo: Instruction tuning of large language models for optimiza-
preprint, 2017. tion code generation,” arXiv preprint, 2024.
[8] S. Ruder, “An overview of gradient descent optimization algorithms,” [36] R. Lange, Y. Tian, and Y. Tang, “Evolution transformer: In-context
arXiv preprint, 2016. evolutionary optimization,” in GECCO, 2024, pp. 575–578.
[9] D. P. Kingma, “Adam: A method for stochastic optimization,” arXiv [37] L. Song, C. Gao, K. Xue, C. Wu, D. Li, J. Hao, Z. Zhang, and C. Qian,
preprint, 2014. “Reinforced in-context black-box optimization,” arXiv preprint, 2024.
[10] D. C. Liu and J. Nocedal, “On the limited memory BFGS method for [38] Y. Chen, M. W. Hoffman, S. G. Colmenarejo, M. Denil, T. P. Lillicrap,
large scale optimization,” Mathematical Programming, vol. 45, no. 1, M. Botvinick, and N. Freitas, “Learning to learn without gradient
pp. 503–528, 1989. descent by gradient descent,” in ICML, 2017.
 19

[39] V. TV, P. Malhotra, J. Narwariya, L. Vig, and G. Shroff, “Meta-learning routing problems with time windows via meta-learning,” Expert Syst.
for black-box optimization,” in ECML PKDD, 2019. Appl., vol. 118, pp. 470–481, 2019.
[40] X. Li, K. Wu, Y. B. Li, X. Zhang, H. Wang, and J. Liu, “GLHF: General [67] Y. Tian, S. Peng, X. Zhang, T. Rodemann, K. C. Tan, and Y. Jin,
learned evolutionary algorithm via hyper functions,” arXiv preprint, “A recommender system for metaheuristic algorithms for continuous
2024. optimization based on deep recurrent neural networks,” TAI, vol. 1,
[41] H. S. Gomes, B. Léger, and C. Gagné, “Meta learning black-box no. 1, pp. 5–18, 2020.
population-based optimizers,” arXiv preprint, 2021. [68] Y. Li, J. Liang, K. Yu, K. Chen, Y. Guo, C. Yue, and L. Zhang,
[42] R. Lange, T. Schaul, Y. Chen, T. Zahavy, V. Dalibard, C. Lu, S. Singh, “Adaptive local landscape feature vector for problem classification and
and S. Flennerhag, “Discovering evolution strategies via meta-black- algorithm selection,” Appl. Soft Comput., vol. 131, p. 109751, 2022.
box optimization,” in GECCO, 2023, pp. 29–30. [69] X. Wu, Y. Zhong, J. Wu, B. Jiang, and K. C. Tan, “Large lan-
[43] R. Lange, T. Schaul, Y. Chen, C. Lu, T. Zahavy, V. Dalibard, and guage model-enhanced algorithm selection: Towards comprehensive
S. Flennerhag, “Discovering attention-based genetic algorithms via algorithm representation,” in IJCAI, 2024.
meta-black-box optimization,” in GECCO, 2023, pp. 929–937. [70] N. Zhu, F. Zhao, and J. Cao, “A hyperheuristic and reinforcement learn-
[44] Q. Guo, R. Wang, J. Guo, B. Li, K. Song, X. Tan, G. Liu, J. Bian, ing guided meta-heuristic algorithm recommendation,” in CSCWD,
and Y. Yang, “Connecting large language models with evolutionary 2024, pp. 1061–1066.
algorithms yields powerful prompt optimizers,” in ICLR, 2024. [71] F. Tahernezhad-Javazm, D. Rankin, N. D. Bois, A. E. Smith, and
[45] R. Lange, Y. Tian, and Y. Tang, “Large language models as evolution D. Coyle, “R2 indicator and deep reinforcement learning enhanced
strategies,” in GECCO, 2024, pp. 579–582. adaptive multi-objective evolutionary algorithm,” arXiv preprint, 2024.
[46] F. Liu, T. Xialiang, M. Yuan, X. Lin, F. Luo, Z. Wang, Z. Lu, [72] G. Cenikj, G. Petelin, and T. Eftimov, “Transoptas: Transformer-based
and Q. Zhang, “Evolution of heuristics: Towards efficient automatic algorithm selection for single-objective optimization,” in GECCO,
algorithm design using large language model,” in ICML, 2024. 2024, pp. 403–406.
[47] A. AhmadiTeshnizi, W. Gao, and M. Udell, “OptiMUS: Scalable opti- [73] H. Samma, C. P. Lim, and J. M. Saleh, “A new reinforcement learning-
mization modeling with (MI) LP solvers and large language models,” based memetic particle swarm optimizer,” Appl. Soft Comput., vol. 43,
in ICML, 2024. pp. 276–297, 2016.
[48] H. Lian, Z. Ma, H. Guo, T. Huang, and Y.-J. Gong, “Rlemmo: [74] A. K. Sadhu, A. Konar, T. Bhattacharjee, and S. Das, “Synergism of
Evolutionary multimodal optimization assisted by deep reinforcement firefly algorithm and q-learning for robot arm path planning,” Swarm
learning,” in GECCO, 2024, pp. 683–693. Evol. Comput., vol. 43, pp. 50–68, 2018.
[49] S.-H. Wu, Y. Huang, X. Wu, L. Feng, Z.-H. Zhan, and K. C. Tan, [75] W. Ning, B. Guo, X. Guo, C. Li, and Y. Yan, “Reinforcement learning
“Learning to transfer for evolutionary multitasking,” arXiv preprint, aided parameter control in multi-objective evolutionary algorithm based
2024. on decomposition,” Prog. Artif. Intell., vol. 7, pp. 385–398, 2018.
[50] Y. Huang, X. Lv, S. Wu, J. Wu, L. Feng, and K. C. Tan, “Advancing [76] Y. Liu, H. Lu, S. Cheng, and Y. Shi, “An adaptive online parameter
automated knowledge transfer in evolutionary multitasking via large control algorithm for particle swarm optimization based on reinforce-
language models,” arXiv preprint, 2024. ment learning,” in CEC, 2019, pp. 815–822.
[77] Z. Li, L. Shi, C. Yue, Z. Shang, and B. Qu, “Differential evolution based
[51] Q. Zhao, Q. Duan, B. Yan, S. Cheng, and Y. Shi, “Automated design
of metaheuristic algorithms: A survey,” TMLR, 2024. on reinforcement learning with fitness ranking for solving multimodal
multiobjective problems,” Swarm Evol. Comput., vol. 49, pp. 234–244,
[52] X. Wu, S.-h. Wu, J. Wu, L. Feng, and K. C. Tan, “Evolutionary
2019.
computation in the era of large language model: Survey and roadmap,”
[78] G. Shala, A. Biedenkapp, N. Awad, S. Adriaensen, M. Lindauer, and
arXiv preprint, 2024.
F. Hutter, “Learning step-size adaptation in CMA-ES,” in PPSN, 2020,
[53] T. Stützle and M. López-Ibáñez, “Automated design of metaheuristic
pp. 691–706.
algorithms,” Handbook of metaheuristics, pp. 541–579, 2019.
[79] K. M. Sallam, S. M. Elsayed, R. K. Chakrabortty, and M. J. Ryan,
[54] M. M. Drugan, “Reinforcement learning versus evolutionary computa- “Evolutionary framework with reinforcement learning-based mutation
tion: A survey on hybrid algorithms,” Swarm Evol. Comput., vol. 44, adaptation,” IEEE Access, vol. 8, pp. 194 045–194 071, 2020.
pp. 228–246, 2019. [80] Y. Huang, W. Li, F. Tian, and X. Meng, “A fitness landscape ruggedness
[55] M. Chernigovskaya, A. Kharitonov, and K. Turowski, “A recent pub- multiobjective differential evolution algorithm with a reinforcement
lications survey on reinforcement learning for selecting parameters of learning strategy,” Appl. Soft Comput., vol. 96, p. 106693, 2020.
meta-heuristic and machine learning algorithms.” in CLOSER, 2023, [81] Z. Hu, W. Gong, and S. Li, “Reinforcement learning-based differential
pp. 236–243. evolution for parameters extraction of photovoltaic models,” Energy
[56] Y. Song, Y. Wu, Y. Guo, R. Yan, P. N. Suganthan, Y. Zhang, Rep., vol. 7, pp. 916–928, 2021.
W. Pedrycz, S. Das, R. Mallipeddi, O. S. Ajani et al., “Reinforce- [82] S. Yin, Y. Liu, G. Gong, H. Lu, and W. Li, “RLEPSO: Reinforcement
ment learning-assisted evolutionary algorithm: A survey and research learning based ensemble particle swarm optimizer,” in ACAI, 2021, pp.
opportunities,” Swarm Evol. Comput., vol. 86, p. 101517, 2024. 1–6.
[57] P. Li, J. Hao, H. Tang, X. Fu, Y. Zhen, and K. Tang, “Bridging [83] T. N. Huynh, D. T. Do, and J. Lee, “Q-learning-based parameter
evolutionary algorithms and reinforcement learning: A comprehensive control in differential evolution for structural optimization,” Appl. Soft
survey on hybrid algorithms,” TEC, 2024. Comput., vol. 107, p. 107464, 2021.
[58] R. S. Sutton, “Reinforcement learning: An introduction,” A Bradford [84] H. Xia, C. Li, S. Zeng, Q. Tan, J. Wang, and S. Yang, “A reinforcement-
Book, 2018. learning-based evolutionary algorithm using solution space clustering
[59] B. Zoph, “Neural architecture search with reinforcement learning,” for multimodal optimization problems,” in CEC, 2021, pp. 1938–1945.
arXiv preprint, 2016. [85] D. Wu and G. G. Wang, “Employing reinforcement learning to enhance
[60] M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, particle swarm optimization methods,” Eng. Optim., vol. 54, no. 2, pp.
T. Schaul, B. Shillingford, and N. De Freitas, “Learning to learn by 329–348, 2022.
gradient descent by gradient descent,” NeurIPS, 2016. [86] F. Wang, X. Wang, and S. Sun, “A reinforcement learning level-based
[61] W. Kool, H. van Hoof, and M. Welling, “Attention, learn to solve particle swarm optimization algorithm for large-scale optimization,”
routing problems!” in ICLR, 2019. Inf. Sci., vol. 602, pp. 298–312, 2022.
[62] R. Storn and K. Price, “Differential evolution–a simple and efficient [87] K. Xue, J. Xu, L. Yuan, M. Li, C. Qian, Z. Zhang, and Y. Yu, “Multi-
heuristic for global optimization over continuous spaces,” J. Glob. agent dynamic algorithm configuration,” NeurIPS, vol. 35, pp. 20 147–
Optim., vol. 11, pp. 341–359, 1997. 20 161, 2022.
[63] K. A. Smith-Miles, “Towards insightful algorithm selection for optimi- [88] Z. Hu and W. Gong, “Constrained evolutionary optimization based on
sation using meta-learning concepts,” in IJCNN, 2008, pp. 4118–4124. reinforcement learning using the objective function and constraints,”
[64] J. Y. Kanda, A. C. de Carvalho, E. R. Hruschka, and C. Soares, “Using KBS, vol. 237, p. 107731, 2022.
meta-learning to recommend meta-heuristics for the traveling salesman [89] Y. Tian, X. Li, H. Ma, X. Zhang, K. C. Tan, and Y. Jin, “Deep rein-
problem,” in ICML, vol. 1, 2011, pp. 346–351. forcement learning based adaptive operator selection for evolutionary
[65] Y. Tian, S. Peng, T. Rodemann, X. Zhang, and Y. Jin, “Automated multi-objective optimization,” TETCI, vol. 7, no. 4, pp. 1051–1064,
selection of evolutionary multi-objective optimization algorithms,” in 2022.
SSCI, 2019, pp. 3225–3232. [90] I. Fister, D. Fister, and I. Fister Jr, “Reinforcement learning-based dif-
[66] A. E. Gutierrez-Rodrı́guez, S. E. Conant-Pablos, J. C. Ortiz-Bayliss, ferential evolution for global optimization,” in Differential Evolution:
and H. Terashima-Marı́n, “Selecting meta-heuristics for solving vehicle From Theory to Practice, 2022, pp. 43–75.
 20

[91] W. Li, P. Liang, B. Sun, Y. Sun, and Y. Huang, “Reinforcement [116] P.-F. Guo, Y.-H. Chen, Y.-D. Tsai, and S.-D. Lin, “Towards optimizing
learning-based particle swarm optimization with neighborhood differ- with large language models,” arXiv preprint, 2023.
ential mutation strategy,” Swarm Evol. Comput., vol. 78, p. 101274, [117] X. Li, K. Wu, X. Zhang, H. Wang, and J. Liu, “B2Opt: Learning
2023. to optimize black-box optimization with little budget,” arXiv preprint,
[92] H. Zhang, J. Sun, T. Bäck, Q. Zhang, and Z. Xu, “Controlling se- 2023.
quential hybrid evolutionary algorithm by q-learning [research frontier] [118] S. Brahmachary, S. M. Joshi, A. Panda, K. Koneripalli, A. K. Sagotra,
[research frontier],” CIM, vol. 18, no. 1, pp. 84–103, 2023. H. Patel, A. Sharma, A. D. Jagtap, and K. Kalyanaraman, “Large
[93] X. Liu, J. Sun, Q. Zhang, Z. Wang, and Z. Xu, “Learning to learn language model-based evolutionary optimizer: Reasoning with elitism,”
evolutionary algorithm: A learnable differential evolution,” TETCI, arXiv preprint, 2024.
vol. 7, no. 6, pp. 1605–1620, 2023. [119] Z. Wang, S. Liu, J. Chen, and K. C. Tan, “Large language model-
[94] S. Yin, M. Jin, H. Lu, G. Gong, W. Mao, G. Chen, and W. Li, aided evolutionary search for constrained multiobjective optimization,”
“Reinforcement-learning-based parameter adaptation method for parti- in ICIC, 2024, pp. 218–230.
cle swarm optimization,” Complex Intell. Syst., vol. 9, no. 5, pp. 5585– [120] W. Yi, R. Qu, L. Jiao, and B. Niu, “Automated design of metaheuristics
5609, 2023. using reinforcement learning within a novel general search framework,”
[95] X. Meng, H. Li, and A. Chen, “Multi-strategy self-learning particle TEC, vol. 27, no. 4, pp. 1072–1084, 2022.
swarm optimization algorithm based on reinforcement learning,” MBE, [121] F. Liu, X. Tong, M. Yuan, and Q. Zhang, “Algorithm evolution using
vol. 20, no. 5, pp. 8498–8530, 2023. large language model,” arXiv preprint, 2023.
[96] Q. Yang, S.-C. Chu, J.-S. Pan, J.-H. Chou, and J. Watada, “Dynamic [122] J. Chen, Z. Ma, H. Guo, Y. Ma, J. Zhang, and Y.-J. Gong, “SYMBOL:
multi-strategy integrated differential evolution algorithm based on re- Generating flexible black-box optimizers through symbolic equation
inforcement learning for optimization problems,” Complex Intell. Syst., learning,” in ICLR, 2024.
vol. 10, no. 2, pp. 1845–1877, 2024. [123] N. van Stein and T. Bäck, “LLaMEA: A large language model
[97] Y. Han, H. Peng, C. Mei, L. Cao, C. Deng, H. Wang, and Z. Wu, evolutionary algorithm for automatically generating metaheuristics,”
“Multi-strategy multi-objective differential evolutionary algorithm with arXiv preprint, 2024.
reinforcement learning,” KBS, vol. 277, p. 110801, 2023. [124] Y. Huang, S. Wu, W. Zhang, J. Wu, L. Feng, and K. C. Tan, “Au-
[98] F. Zhao, Z. Wang, L. Wang, T. Xu, N. Zhu et al., “A multi-agent tonomous multi-objective optimization using large language model,”
reinforcement learning driven artificial bee colony algorithm with the arXiv preprint, 2024.
central controller,” Expert Syst. Appl., vol. 219, p. 119672, 2023. [125] R. Zhang, F. Liu, X. Lin, Z. Wang, Z. Lu, and Q. Zhang, “Under-
[99] Z. Hu, W. Gong, W. Pedrycz, and Y. Li, “Deep reinforcement learning standing the importance of evolutionary search in automated heuristic
assisted co-evolutionary differential evolution for constrained optimiza- design with large language models,” in PPSN, 2024, pp. 185–202.
tion,” Swarm Evol. Comput., vol. 83, p. 101387, 2023. [126] Q. Zhao, T. Liu, B. Yan, Q. Duan, J. Yang, and Y. Shi, “Automated
[100] T. Li, Y. Meng, and L. Tang, “Scheduling of continuous annealing metaheuristic algorithm design with autoregressive learning,” arXiv
with a multi-objective differential evolution algorithm based on deep preprint, 2024.
reinforcement learning,” TASE, vol. 21, no. 2, pp. 1767–1780, 2023. [127] P. Kerschke, H. H. Hoos, F. Neumann, and H. Trautmann, “Automated
[101] L. Peng, Z. Yuan, G. Dai, M. Wang, and Z. Tang, “Reinforcement algorithm selection: Survey and perspectives,” ECJ, vol. 27, no. 1, pp.
learning-based hybrid differential evolution for global optimization of 3–45, 2019.
interplanetary trajectory design,” Swarm Evol. Comput., vol. 81, p.
[128] J. R. Rice, “The algorithm selection problem,” in Advances in Com-
101351, 2023.
puters, 1976, vol. 15, pp. 65–118.
[102] X. Yu, P. Xu, F. Wang, and X. Wang, “Reinforcement learning-
[129] B. Bischl, O. Mersmann, H. Trautmann, and M. Preuß, “Algorithm
based differential evolution algorithm for constrained multi-objective
selection based on exploratory landscape analysis and cost-sensitive
optimization problems,” EAAI, vol. 131, p. 107817, 2024.
learning,” in GECCO, 2012, pp. 313–320.
[103] J. Hong, B. Shen, and A. Pan, “A reinforcement learning-based
neighborhood search operator for multi-modal optimization and its [130] P. Kerschke and H. Trautmann, “Automated algorithm selection on
applications,” Expert Syst. Appl., vol. 246, p. 123150, 2024. continuous black-box problems by combining exploratory landscape
[104] H. Zhang, J. Shi, J. Sun, A. W. Mohamed, and Z. Xu, “A gradient-based analysis and machine learning,” ECJ, vol. 27, no. 1, pp. 99–127, 2019.
method for differential evolution parameter control by smoothing,” in [131] S. Hochreiter, “Long short-term memory,” Neural Comput., vol. 9,
GECCO, 2024, pp. 423–426. no. 8, pp. 1735–1780, 1997.
[105] H. Zhang, J. Sun, T. Bäck, and Z. Xu, “Learning to select the [132] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N.
recombination operator for derivative-free optimization,” Sci. China Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”
Math., pp. 1–24, 2024. NeurIPS, 2017.
[106] Z. Liao, Q. Pang, and Q. Gu, “Differential evolution based on strategy [133] K. Eggensperger, P. Müller, N. Mallik, M. Feurer, R. Sass, A. Klein,
adaptation and deep reinforcement learning for multimodal optimiza- N. Awad, M. Lindauer, and F. Hutter, “HPOBench: A collection of
tion problems,” Swarm Evol. Comput., vol. 87, p. 101568, 2024. reproducible multi-fidelity benchmark problems for HPO,” in NeurIPS,
[107] J. Wang, Y. Zheng, Z. Zhang, H. Peng, and H. Wang, “A novel 2021.
multi-state reinforcement learning-based multi-objective evolutionary [134] Á. Fialho, “Adaptive operator selection for optimization,” Ph.D. dis-
algorithm,” Inf. Sci., vol. 688, p. 121397, 2024. sertation, Université Paris Sud-Paris XI, 2010.
[108] X. Wang, F. Wang, Q. He, and Y. Guo, “A multi-swarm optimizer [135] S. Adriaensen, A. Biedenkapp, G. Shala, N. Awad, T. Eimer, M. Lin-
with a reinforcement learning mechanism for large-scale optimization,” dauer, and F. Hutter, “Automated dynamic algorithm configuration,”
Swarm Evol. Comput., vol. 86, p. 101486, 2024. JAIR, vol. 75, pp. 1633–1699, 2022.
[109] A. Bolufé-Röhler and B. Xu, “Deep reinforcement learning for smart [136] S. Biswas, D. Saha, S. De, A. D. Cobb, S. Das, and B. A. Jalaian,
restarts in exploration-only exploitation-only hybrid metaheuristics,” in “Improving differential evolution through bayesian hyperparameter
MIC, 2024. optimization,” in CEC, 2021, pp. 832–840.
[110] J. Pei, J. Liu, and Y. Mei, “Learning from offline and online experi- [137] J. Brest, M. S. Maučec, and B. Bošković, “Self-adaptive differential
ences: A hybrid adaptive operator selection framework,” in GECCO, evolution algorithm with population size reduction for single objective
2024, pp. 1017–1025. bound-constrained optimization: Algorithm j21,” in CEC, 2021, pp.
[111] S. Chaybouti, L. Dos Santos, C. Malherbe, and A. Virmaux, “Meta- 817–824.
learning of black-box solvers using deep reinforcement learning,” in [138] K. M. Sallam, S. M. Elsayed, R. K. Chakrabortty, and M. J. Ryan,
NeurIPS, 2022. “Improved multi-operator differential evolution algorithm for solving
[112] C. Yang, X. Wang, Y. Lu, H. Liu, Q. V. Le, D. Zhou, and X. Chen, unconstrained problems,” in CEC, 2020, pp. 1–8.
“Large language models as optimizers,” 2024. [139] M. V. Seiler, J. Rook, J. Heins, O. L. Preuß, J. Bossek, and H. Traut-
[113] S. Liu, C. Chen, X. Qu, K. Tang, and Y.-S. Ong, “Large language mann, “Using reinforcement learning for per-instance algorithm con-
models as evolutionary optimizers,” in CEC, 2024, pp. 1–8. figuration on the TSP,” in SSCI, 2023, pp. 361–368.
[114] F. Liu, X. Lin, Z. Wang, S. Yao, X. Tong, M. Yuan, and Q. Zhang, [140] D. Karapetyan and G. Gutin, “Lin–Kernighan heuristic adaptations for
“Large language model for multi-objective evolutionary optimization,” the generalized traveling salesman problem,” EJOR, vol. 208, no. 3,
arXiv preprint, 2023. pp. 221–232, 2011.
[115] J. Lehman, J. Gordon, S. Jain, K. Ndousse, C. Yeh, and K. O. Stan- [141] R. Tanabe and A. S. Fukunaga, “Improving the search performance
ley, “Evolution through large models,” in Handbook of Evolutionary of shade using linear population size reduction,” in CEC, 2014, pp.
Machine Learning, 2023, pp. 331–366. 1658–1665.
 21

[142] S. Wright et al., “The roles of mutation, inbreeding, crossbreeding, and [170] M. Tomassini, L. Vanneschi, P. Collard, and M. Clergue, “A study of
selection in evolution,” 1932. fitness distance correlation as a difficulty measure in genetic program-
[143] R. Lange, Y. Tang, and Y. Tian, “Neuroevobench: Benchmarking ming,” TEC, vol. 13, no. 2, pp. 213–239, 2005.
evolutionary optimizers for deep learning applications,” NeurIPS, 2023. [171] K. M. Malan and A. P. Engelbrecht, “Quantifying ruggedness of
[144] S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and continuous landscapes using entropy,” in CEC, 2009, pp. 1440–1447.
L. Zettlemoyer, “Rethinking the role of demonstrations: What makes [172] G. Merkuryeva and V. Bolshakovs, “Benchmark fitness landscape
in-context learning work?” arXiv preprint, 2022. analysis,” IJSSST, vol. 12, no. 2, pp. 38–45, 2011.
[145] R. Bellman, “A markovian decision process,” JMM, pp. 679–684, 1957. [173] M. Lunacek and D. Whitley, “The dispersion metric and the CMA
[146] C. J. Watkins and P. Dayan, “Q-learning,” Mach. Learn., vol. 8, pp. evolution strategy,” in GECCO, 2006, pp. 477–484.
279–292, 1992. [174] L. Vanneschi, M. Clergue, P. Collard, M. Tomassini, and S. Vérel,
[147] V. Mnih, “Playing atari with deep reinforcement learning,” arXiv “Fitness clouds and problem hardness in genetic programming,” in
preprint, 2013. GECCO, 2004, pp. 690–701.
[148] H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning [175] L. Vanneschi, P. Collard, S. Verel, M. Tomassini, Y. Pirola, and
with double q-learning,” in AAAI, 2016. G. Mauri, “A comprehensive view of fitness landscapes with neutrality
[149] T. Johannink, S. Bahl, A. Nair, J. Luo, A. Kumar, M. Loskyll, J. A. and fitness clouds,” in EuroGP, 2007, pp. 241–250.
Ojea, E. Solowjow, and S. Levine, “Residual reinforcement learning [176] N. Hansen, A. Auger, S. Finck, and R. Ros, “Real-parameter black-
for robot control,” in ICRA, 2019. box optimization benchmarking 2010: Experimental setup,” Ph.D.
[150] R. J. Williams, “Simple statistical gradient-following algorithms for dissertation, INRIA, 2010.
connectionist reinforcement learning,” Mach. Learn., vol. 8, pp. 229– [177] P. N. Suganthan, N. Hansen, J. J. Liang, K. Deb, Y.-P. Chen, A. Auger,
256, 1992. and S. Tiwari, “Problem definitions and evaluation criteria for the CEC
[151] V. Konda and J. Tsitsiklis, “Actor-critic algorithms,” NeurIPS, 1999. 2005 special session on real-parameter optimization,” Tech. Rep., 2005.
[152] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, [178] K. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer
“Proximal policy optimization algorithms,” arXiv preprint, 2017. learning,” J. Big Data, vol. 3, pp. 1–40, 2016.
[153] M. Gao, X. Feng, H. Yu, and X. Li, “An efficient evolutionary [179] Y. Zhang and Q. Yang, “A survey on multi-task learning,” TKDE,
algorithm based on deep reinforcement learning for large-scale sparse vol. 34, no. 12, pp. 5586–5609, 2021.
multiobjective optimization,” Appl. Intell., vol. 53, no. 18, pp. 21 116– [180] M. V. Seiler, P. Kerschke, and H. Trautmann, “Deep-ELA: Deep ex-
21 139, 2023. ploratory landscape analysis with self-supervised pretrained transform-
[154] D. Whitley, T. Starkweather, and C. Bogart, “Genetic algorithms and ers for single-and multi-objective continuous optimization problems,”
neural networks: Optimizing connections and connectivity,” Parallel arXiv preprint, 2024.
Comput., vol. 14, no. 3, pp. 347–361, 1990. [181] Z. Ma, J. Chen, H. Guo, and Y.-J. Gong, “Neural exploratory landscape
[155] N. Hansen, A. Auger, R. Ros, O. Mersmann, T. Tušar, and D. Brock- analysis,” arXiv preprint, 2024.
hoff, “COCO: A platform for comparing continuous optimizers in a
black-box setting,” Optim. Methods Softw., vol. 36, no. 1, pp. 114–
144, 2021.
[156] M. Zinkevich, M. Johanson, M. Bowling, and C. Piccione, “Regret
minimization in games with incomplete information,” NeurIPS, 2007.
[157] A. Maraval, M. Zimmer, A. Grosnit, and H. Bou Ammar, “End-to-
end meta-bayesian optimisation with transformer neural processes,”
NeurIPS, 2024.
[158] A. W. Mohamed, A. A. Hadi, A. K. Mohamed, P. Agrawal, A. Kumar,
and P. N. Suganthan, “Problem definitions and evaluation criteria for
the CEC 2021 special session and competition on single objective
bound constrained numerical optimization,” Tech. Rep., 2021.
[159] U. Škvorc, T. Eftimov, and P. Korošec, “GECCO black-box optimiza-
tion competitions: progress from 2009 to 2018,” in GECCO, 2019, pp.
275–276.
[160] S. Huband, P. Hingston, L. Barone, and L. While, “A review of
multiobjective test problems and a scalable test problem toolkit,” TEC,
vol. 10, no. 5, pp. 477–506, 2006.
[161] X. Li, A. Engelbrecht, and M. G. Epitropakis, “Benchmark functions
for CEC’2013 special session and competition on niching methods for
multimodal function optimization,” Tech. Rep., 2013.
[162] C. Li, S. Yang, T.-T. Nguyen, E. L. Yu, X. Yao, Y. Jin, H. Beyer, and
P. N. Suganthan, “Benchmark generator for CEC 2009 competition on
dynamic optimization,” Tech. Rep., 2008.
[163] X. Li, K. Tang, M. N. Omidvar, Z. Yang, K. Qin, and H. China, “Bench-
mark functions for the CEC 2013 special session and competition on
large-scale global optimization,” Tech. Rep., 2013.
[164] M. A. Muñoz and K. Smith-Miles, “Generating new space-filling test
instances for continuous black-box optimization,” ECJ, vol. 28, no. 3,
pp. 379–404, 2020.
[165] D. Vermetten, F. Ye, T. Bäck, and C. Doerr, “MA-BBOB: A problem
generator for black-box optimization using affine combinations and
shifts,” TELO, 2024.
[166] F. Hutter, M. López-Ibánez, C. Fawcett, M. Lindauer, H. H. Hoos,
K. Leyton-Brown, and T. Stützle, “AClib: A benchmark library for
algorithm configuration,” in LION, 2014, pp. 36–40.
[167] A. Kumar, G. Wu, M. Z. Ali, R. Mallipeddi, P. N. Suganthan, and
S. Das, “A test-suite of non-convex constrained optimization problems
from the real-world and some baseline results,” Swarm Evol. Comput.,
vol. 56, p. 100693, 2020.
[168] C. Doerr, H. Wang, F. Ye, S. Van Rijn, and T. Bäck, “IOHprofiler: A
benchmarking and profiling tool for iterative optimization heuristics,”
arXiv preprint, 2018.
[169] O. Mersmann, B. Bischl, H. Trautmann, M. Preuss, C. Weihs, and
G. Rudolph, “Exploratory landscape analysis,” in GECCO, 2011, pp.
829–836.