Automated Vulnerability Injection in Solidity Smart Contracts:

A Mutation-Based Approach for Benchmark Development

Gerardo Iuliano, Luigi Allocca, Matteo Cicalese, Dario Di Nucci
University of Salerno
Fisciano (SA), Italy
geiuliano@[Link],l.allocca8@[Link],mcicalese@[Link],ddinucci@[Link]

Abstract 1 Introduction
The security of smart contracts is critical in blockchain systems, Smart contracts are self-executing programs operating on blockchain
where even minor vulnerabilities can lead to substantial financial platforms like Ethereum [7]. They enable decentralized applications
arXiv:2504.15948v1 [[Link]] 22 Apr 2025

losses. Researchers proposed several vulnerability detection tools to execute pre-defined terms and conditions autonomously, pro-
evaluated using existing benchmarks. However, most benchmarks moting trust between untrusted parties. With the advancement of
are outdated and focus on a narrow set of vulnerabilities. This blockchain technology, smart contracts have become indispensable
work evaluates whether mutation seeding can effectively inject in various domains, including digital payments and decentralized
vulnerabilities into Solidity-based smart contracts and whether finance [35]. However, their immutable nature, while foundational
state-of-the-art static analysis tools can detect the injected flaws. to the blockchain trust model, presents significant challenges when
We aim to automatically inject vulnerabilities into smart contracts vulnerabilities are discovered [26]. Unlike traditional software, de-
to generate large and wide benchmarks. We propose MuSe, a tool ployed smart contracts cannot be modified or patched, making even
to generate vulnerable smart contracts by leveraging pattern-based minor flaws potentially catastrophic [1].
mutation operators to inject six vulnerability types into real-world The security of smart contracts is critical. Vulnerabilities such as
smart contracts. We analyzed these vulnerable smart contracts us- Reentrancy, Timestamp dependence, Transaction Order Dependence
ing Slither, a static analysis tool, to determine its capacity to identify (TOD), Authorization through [Link], and Unchecked external calls
them and assess their validity. The results show that each vulner- have already led to significant financial losses, exemplified by the
ability has a different injection rate. Not all smart contracts can infamous DAO hack [27]. Researchers and developers have cre-
exhibit some vulnerabilities because they lack the prerequisites for ated a myriad of vulnerability detection tools. Often leveraging
injection. Furthermore, static analysis tools fail to detect all vulner- static or dynamic analysis, these tools aim to identify and mitigate
abilities injected using pattern-based mutations, underscoring the potential defects before deployment and, in some cases, on-chain.
need for enhancements in static analyzers and demonstrating that Despite their advancements, these tools are not infallible. False
benchmarks generated by mutation seeding tools can improve the positives [30], false negatives, and the inability to detect complex
evaluation of detection tools. vulnerability patterns limit their effectiveness, underscoring the
need for robust evaluation.
CCS Concepts One critical barrier to improving detection tools is the lack of
• Software and its engineering → Software verification and comprehensive and diverse benchmarks [13]. Existing datasets are
validation; • Security and privacy → Software security engi- often outdated, limited in size, or narrowly focused on specific
neering. vulnerabilities, leaving many tools untested against realistic sce-
narios. The most commonly used benchmarks [15] comprise smart
Keywords contracts written using the old Solidity version and affected by a
subset of the known vulnerability. Other datasets often contain
Vulnerability, Smart Contract, Mutation Testing, Benchmark
simplistic toy contracts [39], like the SWC Registry. In addition,
ACM Reference Format: about 96% of smart contracts present in datasets are involved in no
Gerardo Iuliano, Luigi Allocca, Matteo Cicalese, Dario Di Nucci. 2018. Auto- more than five transactions [9]. Ren et al. [32] highlighted that tools
mated Vulnerability Injection in Solidity Smart Contracts: A Mutation-Based should be evaluated using a comprehensive benchmark suite that
Approach for Benchmark Development. In Proceedings of The 29th Inter- integrates multiple vulnerability types. The absence of high-quality
national Conference on Evaluation and Assessment in Software Engineering
and updated benchmarks hinders progress in the field by affecting
(EASE 2025). ACM, New York, NY, USA, 11 pages. [Link]
the ability to validate and improve detection tools.
XXXXXXX
To address this gap, we propose MuSe, a mutation-based tool
Permission to make digital or hard copies of all or part of this work for personal or based on SuMo [2] to generate benchmarks by injecting vulner-
classroom use is granted without fee provided that copies are not made or distributed abilities into smart contracts. By leveraging mutation operators
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the designed around known vulnerability patterns, our approach sys-
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or tematically introduces faults into realistic scenarios, enabling the
republish, to post on servers or to redistribute to lists, requires prior specific permission generation of contracts with vulnerabilities placed in both typi-
and/or a fee. Request permissions from permissions@[Link].
EASE 2025, Istanbul, Türkiye cal and unconventional yet valid locations, challenging detection
© 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. tools to expand their scope and improve accuracy. Such versatil-
ACM ISBN 978-1-4503-XXXX-X/2018/06 ity allows for evaluating detection tools against various scenarios,
[Link]
EASE 2025, 17–20 June, 2025, Istanbul, Türkiye Iuliano et al.

including edge cases. The strength of our approach lies in its foun- 2.2 Smart Contract Vulnerabilities
dation on pattern-based mutation operators. These operators are Ethereum smart contracts are prone to various vulnerabilities unique
designed to inject vulnerabilities wherever their corresponding pat- to blockchain technology. Reentrancy is one of the most criti-
terns are identified, ensuring consistency and extensibility; as new cal [24, 25], as evidenced by the infamous DAO hack, where an
vulnerabilities are identified, corresponding operators can be added attacker repeatedly called back into a contract to drain its funds.
with minimal effort. We manually validated mutation operators Another major issue is the misuse of transaction origin for autho-
and evaluated the generated benchmarks using Slither [16], a rization, which attackers can easily spoof to gain unauthorized
state-of-the-art static analysis tool for smart contracts. access. Timestamp manipulation by miners is another concern, al-
The results reveal that each vulnerability has a different injec- lowing them to alter timestamps and compromise the security of
tion rate, and successful injection depends on whether the smart critical contract functions [22, 28]. Transaction-ordering depen-
contract satisfies the necessary preconditions to adhere to vulnera- dence (TOD), where the order of transactions is unpredictable, can
bility patterns. Detection outcomes reveal significant limitations of be exploited to unfairly manipulate outcomes, such as reducing
Slither in identifying vulnerabilities, especially when they are in- rewards before submitting a valid solution [14, 34, 37]. External
jected into unconventional or unexpected locations. These findings calls also pose significant risks. Attackers can exploit these calls to
underscore the weaknesses of current static analysis tools and high- execute malicious code. Furthermore, failing to handle a function
light the pressing need for more advanced detection techniques. return value properly may enable attackers to drain contract bal-
Moreover, MuSe allows to increase the benchmark size of above ances. Denial of Service (DoS) attacks can arise in several ways [33],
840%. In conclusion, our study identifies the gaps in the static ana- such as through costly external calls or inefficient looping behavior.
lyzer, offering a tangible solution for researchers and practitioners Smart contract development differs fundamentally from traditional
to enhance or generate new benchmarks to evaluate their detection software programming. Vulnerabilities in smart contracts deployed
tools. Our work provides the following contributions: on public blockchains are particularly challenging to fix due to the
• MuSe, a mutation seeding tool to generate benchmarks in- immutable nature of blockchain systems. While some traditional se-
jecting vulnerabilities in Solidity smart contracts using pattern- curity techniques are applicable, smart contracts introduce unique
based mutation operators; challenges, and many vulnerabilities arise from the distinctive char-
• an enhanced version of the dataset smartbugs-wild contain- acteristics of blockchain technology.
ing 350,493 vulnerable smart contracts;
• a list of weaknesses that affect the capabilities of Slither in 2.3 Mutation Testing
detecting vulnerabilities.
Mutation testing is a software testing technique to evaluate the ef-
Paper Structure. This paper is organized as follows. Section 2 fectiveness of test cases by intentionally introducing small changes,
establishes the background and reviews related work. Section 3 called mutants, into the source code to simulate potential faults or
details the research method, including research questions and the errors. The primary goal is to assess whether the existing test cases
mutation-based approach. Section 4 presents the experimental re- can detect these changes, thereby measuring the fault-detection
sults, while Section 5 discusses key findings, implications, and their capability of a test suite [21, 29]. This process ensures software
relevance. Section 6 addresses threats to validity, and Section 7 is rigorously validated, improving its reliability and reducing the
concludes the paper and provides future research directions. likelihood of undetected faults in production.
Mutation testing in Solidity applies the same principles as tradi-
2 Background and Related Work tional mutation testing. Still, it focuses on the unique characteristics
of smart contracts, injecting Solidity-specific faults to evaluate the
This section introduces smart contracts, their vulnerabilities, and
effectiveness of vulnerability detection and test suite robustness.
mutation testing. Furthermore, we provide some related work that
In the literature, some tools have been proposed for this purpose.
is relevant to our study.
Chapman et al. proposed Deviant [8], a mutation testing tool
designed for Solidity smart contracts. It automatically generates
2.1 Smart Contracts mutated versions of a given Solidity project and runs them against
Smart contracts are self-executing programs designed to automati- existing test suites to assess their effectiveness. Deviant includes
cally enforce terms and conditions between untrusted parties [23]. mutation operators that cover Solidity-specific features based on a
Initially envisioned as a way to automate legal contracts, the rise Solidity fault taxonomy and traditional programming constructs
of blockchain technology has transformed smart contracts into to simulate faults. The authors used Deviant to evaluate the test
scripts that execute synchronously across nodes in a distributed effectiveness of three Solidity projects. Their findings show that
ledger [40]. On the Ethereum blockchain, smart contracts run achieving high statement and branch coverage in Solidity does not
within the Ethereum Virtual Machine (EVM), a Turing-complete, guarantee strong code quality. This study provides valuable insights
stack-based virtual machine that ensures isolated contract code for Solidity developers, emphasizing the need for more rigorous
execution. A smart contract is identified by a unique address, pri- testing to minimize financial risks. Ivanova and Khritankov pre-
vate storage, and a balance in Ether, and it contains executable sented RegularMutator [20], a tool for improving the reliability
code. When a transaction is sent to a contract address, it triggers of smart contracts written using Solidity language. RegularMu-
the contract functionality, providing invocation data and paying tator implements language-specific operators that correspond to
transaction fees using Gas [23]. common errors made during the development of smart contracts.
Automated Vulnerability Injection in Solidity Smart Contracts:
A Mutation-Based Approach for Benchmark Development EASE 2025, 17–20 June, 2025, Istanbul, Türkiye

Injection of mutations in program code is implemented using regu- version, a version with extensive protections, and a version with-
lar expressions. The tool demonstrated its effectiveness in testing out protections. Faults were injected into the unprotected versions,
large-scale smart contract projects. The study concluded that muta- resulting in 651 faulty variants. The findings revealed that formal
tion analysis provides a more reliable measure of test suite quality verification and runtime protections complement built-in platform
than traditional test line coverage. Barboni et al. proposed SuMo [3], checks but cannot detect all faults. Our study focused on a signifi-
a mutation testing tool designed for Solidity smart contracts, in- cantly larger dataset of faulty smart contracts and targeted critical
corporating 25 Solidity-specific mutation operators alongside 19 smart contract vulnerabilities. Additionally, the goal was distinct,
traditional ones. It enables mutation testing on Solidity projects to focusing on generating benchmarks that can be used to validate
assess test effectiveness. SuMo was later extended by ReSuMo [4], and improve the effectiveness of vulnerability detection tools.
which introduces a regression mutation testing approach. ReSuMo Regarding benchmark generation, other techniques have been
employs a static, file-level technique to selectively mutate a sub- explored in the literature, such as analyzing audit reports and lever-
set of smart contracts and rerun only relevant test cases during aging large language models (LLMs). On the one hand, Zheng et
regression testing. After each mutation testing run, ReSuMo in- al. [39] created a large-scale dataset of SWC weaknesses from real-
crementally updates its results by leveraging test outcomes from world DApp projects. They recruited 22 participants to analyze
previous program revisions, improving efficiency and reducing 1,199 open-source audit reports from 29 security teams, identifying
redundant computations. 9,154 weaknesses. Their work resulted in two distinct datasets. The
DAppSCAN-Source dataset contains 39,904 Solidity files with 1,618
SWC weaknesses. However, these files may not be directly com-
2.4 Related Work pilable for automated analysis. To address this, the authors devel-
Bug injection is a testing method widely studied in traditional oped a tool that automatically identifies dependency relationships
software programs; however, only a few studies have addressed its within DApp projects and resolves missing public libraries. The
application to smart contracts using mutation security testing. DAppSCAN-Bytecode dataset includes 6,665 compiled smart con-
Ghaleb and Pattabiraman proposed SolidiFI [17], an automated tracts containing 888 SWC weaknesses. Evaluation results showed
and systematic approach to evaluate static analysis tools. The tool that existing detection tools perform poorly on these datasets, high-
injects bugs into a smart contract to introduce the targeted vulner- lighting the need for future research to focus on real-world smart
abilities and then checks the generated buggy contracts using the contract datasets rather than simplistic toy contracts. On the other
static analysis tools. The tool injects bugs by adding vulnerable hand, Daspe et al. [11] utilized large language models (LLMs) to
code snippets, code transformations, and weakening security mech- generate a dataset of Solidity smart contracts. To guide the LLM
anisms. We extended the code transformation approach, focusing during the generation process, they adopted an approach inspired
on more vulnerabilities and a larger dataset to experiment. We also by Test-Driven Development (TDD) [5]. Each prompt was submit-
validated the mutation seeding tool to provide a tool to generate ted to an LLM, producing Solidity code that was then parsed. The
new vulnerable smart contracts starting from an initial set. Solidity compiler verified the syntax, and if the contract compiled
Chu et al. [10] introduced SGDL (Smart Contract Vulnerability successfully, Slither was applied for static analysis to detect vul-
Generation with Deep Learning), an approach to create authentic nerabilities. Next, they created a project containing the generated
and diverse vulnerability datasets for smart contracts. SGDL com- contract and functional tests, which were then executed. They col-
bines generative adversarial networks (GANs) with static analysis lected the compilation status, vulnerability report, and functional
to extract syntactic and semantic information from contracts. Us- test results for each prompt. After multiple generations, they con-
ing this information, it generates realistic vulnerability fragments ducted an evaluation based on prompt complexity and the model
and injects them into smart contracts via an abstract syntax tree, used. The study found that LLMs struggled with increased complex-
ensuring syntactic correctness. The approach depends on a labeled ity, demonstrating low accuracy as contract intricacy grew. Most
dataset and researchers’ expertise in vulnerabilities to train the compilation errors arose from incorrect type usage, like strings and
GANs. However, comprehensive datasets for various vulnerabilities arrays, and issues related to the payable/call pattern.
remain scarce or are not openly accessible for academic research. The work described above highlights the challenges faced and
Consequently, SGDL focus is limited to specific vulnerabilities. In the different approaches used in literature to generate benchmarks.
contrast, our mutation-based approaches rely on predefined pat- Our work bridges some of the gaps by offering a possible solution.
terns and manually designed mutation operators. New operators
do not depend on datasets but on vulnerability patterns, making
the approach easy to extend to other vulnerabilities.
Hajdu et al. [18] conducted a study using software-implemented
fault injection (SWIFI ) to evaluate the dependability of permis- 3 Research Method
sioned blockchain systems in the presence of faulty smart contracts.
The following section presents the details of the study, highlighting
They introduced general software and blockchain-specific faults
the main goal and its related research questions.
into smart contract code to assess their impact on system reliability
and integrity. They also investigated the effectiveness of formal ◎ Goal of the study.
verification and runtime protection mechanisms in detecting and Our goal is to automatically inject vulnerability in smart con-
mitigating these faults. The authors used Hyperledger Fabric and tracts to generate large and wide benchmarks that researchers
evaluated 15 smart contracts, each tested in three versions: a base and developers can use to improve detection tool evaluation.
EASE 2025, 17–20 June, 2025, Istanbul, Türkiye Iuliano et al.

Figure 1: Summary of the Research Method.

To achieve this goal, we developed MuSe, a tool to generate ü RQ2 . How can the mutants injected by MuSe be detected
vulnerable smart contracts that are challenging to detect automati- through static analysis?
cally. Inspired by mutation testing [21, 29], we extended the SuMo
mutation testing tool [2] to mutate Solidity smart contracts into 𝑅𝑄 2 analyzes the performance of static analyzer when detecting
vulnerable mutants. To this end, we implemented mutation oper- vulnerabilities injected into contracts. The goal of applying various
ators designed to inject known vulnerabilities by modifying the patterns to introduce vulnerabilities is twofold. On the one hand, we
smart contracts appropriately. Afterward, we analyzed to what aim to identify whether the injected vulnerabilities are challenging
extent the injected vulnerabilities are challenging to detect by ob- to detect. On the other hand, we want to evaluate whether a static
serving how well static analyzers can detect them. We leveraged analysis tool can identify these specific patterns, highlighting its
Slither [16], a popular static analyzer to detect vulnerabilities in strengths and weaknesses.
Solidity, and compared the results achieved by running it before
and after the mutation phase, assessing the presence of the injected 3.1 Data Collection
vulnerabilities and the performance of Slither in detecting them.
To answer our research questions, we used the real-world dataset
Our motivation is the critical role that smart contract security
smartbugs-wild1 , which contains 47,398 smart contracts extracted
plays in the blockchain and the limitations of existing vulnerability
from the Ethereum network that have at least one transaction. We
detection tools in handling complex vulnerabilities. By combining
ran Slither using SmartBugs [12], which allows us to parallelize its
mutation security testing with static analysis, this work seeks to
execution using several Docker images and easily parse the results.
provide empirical evidence of the strengths and weaknesses of
tools like Slither, offering insights that can drive the development
3.2 Mutation Operators
of more robust security solutions and methodologies for smart
contracts. Figure 1 depicts our research method. We extended SuMo [2], a mutation testing tool, by creating new
As shown in Table 1, we selected six vulnerabilities [19, 31, 36] mutation operators focusing on security. The tool uses the solidity-
to inject based on their relevance in the literature and the ability of parser-antlr 2 , a parser built from a robust ANTLR4 grammar, which
Slither to detect them with at least medium confidence: Unchecked generates an Abstract Syntax Tree of the code based on the Solidity
call return value, Unchecked send, Authentication through [Link], grammar. We implement a mutation operator for each vulnerability
Delegatecall to untrusted callee, and Unused return. These vulner- to inject and leverage the parser to identify injection patterns where
abilities are among the top 15 most discussed in the literature, as the vulnerabilities can be injected. Table 1 maps the implemented
highlighted in the work of Zaazaa Oualid and El Bakkali Hanan [38]. mutation operators, the vulnerability they inject, and the detector
Based on our goal, we formulate these research questions (RQs): Slither uses to identify them.
UC Operator. To inject the Unchecked low-level call return value,
ü RQ1 . To what extent are the mutation operators implemented
we identify the possible instructions or statements to mutate. We
in MuSe generalizable?
identified all the low-level call functions. We mutated the contract
by removing the controls whenever the return value of the functions
𝑅𝑄 1 allows us to assess the feasibility of introducing vulnerabili- was checked using the require function or an if statement.
ties into real-world smart contracts and understand the generaliz-
1 [Link]
ability of the mutation operators to inject a vulnerability.
2 [Link]
Automated Vulnerability Injection in Solidity Smart Contracts:
A Mutation-Based Approach for Benchmark Development EASE 2025, 17–20 June, 2025, Istanbul, Türkiye

Table 1: Vulnerabilities injected into smart contracts by MuSe.

Vulnerability Description Slither’s Detector Operator

Unchecked low-level call return value Low-level calls return false on failure instead of throwing exceptions, risking unchecked-lowlevel UC
critical vulnerabilities if unchecked.
Unchecked send The send function returns false on failure without throwing an exception, risking unchecked-send US
vulnerabilities if unchecked.
Authentication via [Link] Using [Link] for authorization risks vulnerabilities if an authorized account tx-origin TX
interacts with a malicious contract.
Unused return The return value of an external call is not stored in a local or state variable. unused-return UR
Multiple calls in a loop Calls inside a loop might lead to a denial-of-service attack. calls-loop CL
Delegatecall to untrusted callee Delegatecall executes the code at the target address in the context of the calling controlled-delegatecall DTU
contract. It allows a SC to load code dynamically from a different address.

1 // Before UC mutation 1 // Before UR mutation

2 function withdraw ( uint amount ) public { 2 function addNumbers ( uint256 a , uint256 b ) public {
3 require ( msg . sender . call . value ( amount ) () ) ; 3 c = SafeMath . add (a , b ) ;
4 } 4 }
5 // After UC mutation 5 // After UR mutation
6 function withdraw ( uint amount ) public { 6 function addNumbers ( uint256 a , uint256 b ) public {
7 msg . sender . call . value ( amount ) () ; 7 SafeMath . add (a , b ) ;
8 } 8 }

CL Operator. Multiple calls in a loop were injected by wrapping

US Operator. To inject Unchecked send, we identified all the send call, send, and transfer functions in a statement of 1,000 loops
functions and removed any control on the return value. implemented using the “for” construct.

1 // Before CL mutation
2 function payMember ( address payable member ) public {
1 // Before US mutation
3 require ( member . send (0.1 ether ) ;
2 function sendEth ( address payable giftee ) public {
4 }
3 if (! giftee . send (1 ether ) ) {
5 // After CL mutation
4 revert (" Send failed " ) ;
6 function payMember ( address payable member ) public {
5 }
7 for ( uint256 i = 1; i <= 5; i ++) {
6 }
8 require ( member . send (0.1 ether ) ;
7 // After US mutation
9 }
8 function sendEth ( address payable giftee ) public {
10 }
9 giftee . send (1 ether )
10 }
DTU Operator. Delegatecall to untrusted callee was injected by
introducing a new address variable and a new function that allows
TX Operator. Authentication through [Link] was injected sub- users to replace the address variable with another. Each use of
stituting the [Link] variable with [Link]. We mutate the delegatecall was mutated, replacing the address used to delegate
contracts when the [Link] variable is used in a binary opera- with the new personalizable address, which can be set to malicious.
tion like “==” to check the ownership of the contract or a specific
address having some privileges or access to the asset. 1 // Before DTU mutation
2 function setFalseValue ( address _address ) public {
3 require ( _address . delegatecall (
4 abi . encodeWithSignature ( " setFalse ( uint256 )")));
1 // Before TX mutation 5 }
2 modifier onlyOwner () { 6 // After DTU mutation
3 require ( msg . sender == owner , " No owner " ) ; _ ; 7 address public delegate ;
4 } 8 function setDelegate ( address _delegate ) public {
5 // After TX mutation 9 delegate = _delegate ;
6 modifier onlyOwner () { 10 }
7 require ( tx . origin == owner , " No owner " ) ; _ ; 11 function setFalseValue ( address _address ) public {
8 } 12 require ( delegate . delegatecall (
13 abi . encodeWithSignature ( " setFalse ( uint256 )")));
14 }

UR Operator. Unused return was injected in two cases. First, when

a binary operation like “=” assigns the return value of a function to 3.3 MuSe Validation
a variable. Second, when a variable is declared and then initialized To ensure the validity of our results, we manually validated MuSe
with the return value of a function. In both cases, we removed the on a statistically significant subset of mutated smart contracts with
left side of the assignment and left the call function. a 95% confidence level and a 5% margin error. The tool generated
EASE 2025, 17–20 June, 2025, Istanbul, Türkiye Iuliano et al.

350,716 mutated contracts, from which we randomly selected a each operator. MuSe mutated 41,337 out of 47,398 smart contracts,
subset of 384 smart contracts, representing a statistically significant about 87% of cases. The remaining 6,061 smart contracts were not
sample size. Our validation process involved manually analyzing mutated for two reasons: (i) the absence of any patterns used by
each contract to verify whether SuMo injected the intended vulner- mutation operators in 5,990 smart contracts and (ii) the invalid
abilities correctly. The procedure consisted of these steps: content of the files for 71 of them, e.g., a JSON representation of
(1) Compilation for Syntactical Correctness. Each selected smart the smart contract instead of well-formatted Solidity code.
contract was compiled to ensure its syntactical correctness.
(2) Comparison with SuMo Logs. We compared the logs provided 4.1 RQ1. To what extent are the mutation
by SuMo, which detail the applied mutation type and the operators implemented in MuSe
lines of code affected, against the corresponding mutated generalizable?
smart contracts. To answer RQ1 , we observed the number of smart contracts that
(3) Pattern Adherence Verification. We verified that the points could be correctly mutated. We aimed to understand how many
where vulnerabilities were injected adhered to the patterns pattern occurrences applied by each mutation operator could be
defined by the mutational operator. This step ensured that injected in real-world scenarios.
SuMo identified the correct lines of code and statements for
injecting vulnerabilities. Table 2: Results for mutation operators ordered by injection
(4) Modification Assessment. We examined SuMo’s modifications rate, total number of generated mutants, and average injec-
or additions to the original code to determine whether the tion rate of MuSe.
injected vulnerabilities were accurately implemented.
A mutation was marked as correctly injected if (i) the mutated Operator # Mutated SCs # Mutants Injection Rate
contract contained the new lines of code introducing the target UR 33,910 213,912 71.50%
vulnerability or (ii) existing lines of code were altered to render the TX 32,250 65,825 68.00%
smart contract vulnerable to the intended issue. CL 26,604 61,687 56.00%
The validation results show that our tool failed to inject vulnera- UC 4,094 4,992 8.60%
bilities in 20 out of 384 smart contracts, or 5.21% of cases. The main US 2,248 3,928 4.70%
causes of these failures are exceptional cases that the mutational DTU 113 149 0.23%
operator does not adequately handle. One common issue arises - - 350,493 34.83%
when the mutated statement contains a semicolon (“;”). Strings
including a semicolon within the mutated statement can interfere
with the operator logic, leading to unintended code truncation. As a As shown in Table 2, the most injectable vulnerability is the
result, the generated mutant may have incorrect syntax, rendering unused return, with an injection rate of 71.5%. The patterns used
the contract uncompilable. Another cause of injection failure is to inject vulnerabilities are not only common but also frequently
conflicts between the scopes of contract variables and the variables encountered in smart contracts, highlighting their prevalence in
introduced by the mutation. For example, the CL operator injects a typical contract design. On average, each smart contract contains
for loop that uses the variable uint i for iteration. If the mutation six occurrences of these patterns, reflecting their foundational role
is applied to a statement declaring a uint i variable, the compiler in contract development. Patterns such as assignments, declarations,
cannot differentiate between the two variables, leading to a com- and initializations are essential building blocks in smart contract
pilation error. In all other cases, the mutation process successfully programming. However, their prevalence also increases the likeli-
injects the vulnerability without issues, producing valid mutants. hood of vulnerabilities arising from improper or unintended usage.
The second vulnerability, authorization via [Link], is injectable
3.4 Replication Package in a real-world scenario in 68% of cases. The high injection rate
respects the frequency of the pattern of the TX operator in the
We have made MuSe publicly available on GitHub3 , allowing re- smart contracts. As described in Section 3, the TX operator mu-
searchers and practitioners to replicate our study or utilize the tool tates a contract when the variable [Link] is used to check the
for their purposes. Additionally, we have uploaded the sample used ownership of a contract or the privilege of an address on the asset.
to manually validate the mutation operators.4 The injection rate for multiple calls in a loop is 56%, showing
only just over half of the contracts involve the use of a call, send,
4 Empirical Results or transfer function. The moderate injection rate highlights that
We ran MuSe on the SmartBugs-wild dataset, applying the six these functions are commonly employed in contracts but not exces-
previously described mutation operators to each contract. Starting sively frequent. In addition, 8% of the unmutated smart contracts
from 47,398 smart contracts, we generated 350,493 vulnerable ones. already contained the vulnerability.
MuSe mutates a contract each time it matches the pattern of a Unchecked low-level call return value and unchecked send follow
mutation operator. A contract could exhibit more than a pattern. with injection rates of 8.6% and 4.7%, respectively. The frequency of
Table 2 shows the number of mutants generated by each operator a call function is almost double that of a send function. It is gener-
and highlights the number of contracts suitable to be mutated by ally better to use call function than send but with some important
3 [Link] security and implementation considerations. The call function is
4 [Link] more flexible and allows specifying the amount of gas sent and calls
Automated Vulnerability Injection in Solidity Smart Contracts:
A Mutation-Based Approach for Benchmark Development EASE 2025, 17–20 June, 2025, Istanbul, Türkiye

with data. Nevertheless, it is more vulnerable to reentrancy attacks As shown in Table 3, the results achieved by Slither against
because it enables the receiving contract to execute arbitrary code. the unchecked low-level call return value (UC) and unchecked send
Finally, delegatecall to untrusted callee has the lowest injection (US) vulnerabilities are surprisingly high. The tool detected all mu-
rate. The use of the delegatecall function is relatively rare but tants with the injected vulnerability, showing a recall value equal to
not negligible. It is mainly limited to specific use cases that require 1.00 in both cases. The two vulnerabilities are conceptually similar,
advanced behavior, like implementing the Proxy pattern. which shows that Slither implements strong detectors to check
By analyzing the injection rate, it is possible to see that some whether the return values of the call and send functions are handled
contracts do not have the necessary conditions to inject a given correctly. Slither also performs very well in detecting multiple
vulnerability. Contracts that lack specific patterns, constructs, or calls in a loop (CL), with a recall of 0.81. While the detection is
known Solidity functions are intrinsically safe from vulnerabilities strong, a noticeable portion of vulnerabilities remains undetected,
that try to exploit these elements. Overall, mutation operators can indicating room for improvement in the detection mechanism of
increase the size of a dataset by 840% by creating new vulnerable this vulnerability. Performance slows down on unused return (UR)
versions of smart contracts. with a recall value of 0.63, suggesting that the detection mechanism
of these vulnerabilities might be less robust or prone to specific
¤ RQ1 Summary. The results highlight that the patterns limitations. Performance deteriorates on authorization via [Link]
needed to inject “unused return” (71.5%), “authorization via (TX), 0.33 of recall, pointing out significant gaps in detection capa-
[Link]” (68%), and “multiple calls in a loop” (56%) are com- bilities for this category. The worst result is detecting delegatecall
mon, whereas those related to “unchecked low-level call return to untrusted callee (DTU) with a recall value of 0.10.
values” (8.6%) and “unchecked send” (4.7%) are less prevalent. Overall, Slither achieved a recall value of 0.597. The recall
The pattern for “delegatecall to untrusted callee” is rare (0.23%) value indicates that while Slither effectively identifies certain
because the delegatecall function is used infrequently and only vulnerabilities, it fails to detect a significant portion (40.2%) of the
in specific locations. injected vulnerabilities. The result underscores the need to improve
static analysis tools or complement them with additional detection
techniques to enhance their accuracy and reduce false negatives.
4.2 RQ2. How can the mutants injected by MuSe
be detected through static analysis? ¤ RQ2 Summary. Slither performs very well in some cases
To answer RQ2 , first, we performed an initial detection with Slither but inconsistently across vulnerability types. While it excels in
on the smartbugs-wild dataset and collected the findings for each detecting simple vulnerabilities like unchecked low-level call
contract to have a baseline. Then, we ran Slither on the mutants return value and unchecked send, it struggles with more complex
generated by MuSe. Under the assumption that the mutation oper- vulnerabilities like authorization via [Link] and delegatecall
ator correctly injects the vulnerability, we compared the detection to untrusted callee. Finally, Slither detected the 59.7% of the
results before and after the mutation. Slither correctly detects a injected vulnerabilities using MuSe.
mutant if it is labeled as vulnerable to the type of injected vulnera-
bility. In the case where the contract is already vulnerable and has
been mutated, we checked whether, in addition to the pre-existing 5 Discussions and Limitations
vulnerability, the injected vulnerability had also been detected by In this section, we analyze the false negatives resulting from run-
analyzing the lines of code related to the injected vulnerability. ning Slither on the mutated smart contracts. We also discuss the
Slither successfully analyzed 335,234 mutants out of the 350,493 limitations of the study, offering insights about our mutation-based
generated. Similarly, the execution on the smartbugs-wild dataset approach to injecting vulnerabilities.
failed on 2,700 smart contracts out of 47,398. After excluding the
failed executions, we mapped the results achieved by Slither on the
original contracts with the mutated ones, if present, and compared 5.1 False Negative Analysis
the results for further analysis. We analyzed false negatives to extract information about the errors
achieved by Slither in detecting vulnerabilities. The way we con-
ducted the experiment allows us to extract only true positives (TP)
and false negatives (FN). The mutational operators, validated as
Table 3: Detection rate of injected vulnerability and overall
described in Section 3, inject the target vulnerability. Having Slither
performance of Slither on the six considered vulnerabilities.
results before and after the mutation, we can analyze whether the
tool detects the injected vulnerability (TP) or fails detection (FN).
Vulnerability TP FN Recall FNR
UC 4,876 0 1.000 0.000 Authorization via [Link] (TX). For this vulnerability, the FNR
US 3,570 0 1.000 0.000 is 0.663. We found some patterns that Slither does not check when
CL 45,261 10,563 0.810 0.189 detecting this vulnerability type. The most relevant and alarming
UR 124,858 81,184 0.605 0.394 is related to the Solidity modifier. A Solidity modifier is a reusable
TX 21,765 42,937 0.336 0.663 function that encapsulates and enforces reusable logic, such as ac-
DTU 15 134 0.100 0.899 cess control or precondition checks, simplifying code and improving
- 200,345 134,818 0.597 0.402 maintainability. Figure 2 shows the incorrect implementation of a
EASE 2025, 17–20 June, 2025, Istanbul, Türkiye Iuliano et al.

modifier used to check the contract owner using [Link] and

that Slither cannot label as vulnerable. Figure 4: Two examples of unused return.
(a) Unused return detected at line 10.
Figure 2: Incorrect modifier to restrict owner’s access.
1 library SafeMath {
2 function add ( uint256 a , uint256 b ) internal pure
1 modifier onlyOwner () { returns ( uint256 ) {
2 require ( tx . origin == owner ) ; _ ; 3 uint256 c = a + b ;
3 } 4 require ( c >= a , " addition overflow " ) ;
5 return c ;
6 }
7 }
Another scenario that Slither fails to detect is when the clause 8 contract SafeMathExample {
to check the contract ownership is composed of several conditions 9 function addNumbers ( uint256 a , uint256 b ) public {
combined using AND/OR operators. For example, Figure 3 shows 10 SafeMath . add (a , b ) ;
11 }
the addPartner function, which allows the caller to add a new 12 }
partner. The clause in the require function ensures that only
authorized users (_dev or _owner) can call this function. The TX
(b) Unused return not detected at line 10.
operator mutated the second occurrence of [Link], replacing
it with [Link]. Using [Link] for authorization introduces
1 contract SafeMath {
a security risk because it refers to the address that initiated the 2 function add ( uint256 a , uint256 b ) internal pure
transaction, even if there were intermediate contract calls. returns ( uint256 ) {
3 uint256 c = a + b ;
4 require ( c >= a , " addition overflow " ) ;
Figure 3: Clause with two conditions combined using OR. 5 return c ;
6 }
7 }
1 function addPartner ( address _partner ) public { 8 contract SafeMathExample is SafeMath {
2 require (( msg . sender == _dev ) || ( tx . origin == 9 function addNumbers ( uint256 a , uint256 b ) public {
_owner )) ; 10 SafeMath . add (a , b ) ;
3 exchangePartners [ _partner ] = true ; 11 }
4 } 12 }

is catastrophic if misused in the Proxy pattern. The Proxy pattern

Unused return (UR). Slither achieved a false negative rate of
uses delegatecall to separate state and logic, allowing updates
0.394 on this vulnerability. Analysis of false negatives produced
without losing contract data.
an interesting finding. The unused return is correctly detected
by Slither when the function being called in the contract is a
library function; see Figure 4a. However, the detection fails when
the function called is inherited from a contract; see Figure 4b. Figure 5: Delegatecall to untrusted callee.

Multiple calls in a loop (CL). Slither achieved a false negative 1 address public delegate ;
rate of 0.189 on this vulnerability. The mutated smart contracts are 2 function setDelegate ( address _delegate ) public {
characterized by several aspects, like function visibility and the 3 delegate = _delegate ;
presence of modifiers. In addition, the mutation can be injected 4 }
5 function upgradeAndCall ( address implementation ,
into the function bodies or in-depth, e.g., nested into an if state- bytes calldata data ) external payable ifAdmin {
ment. Finally, the statement that undergoes the mutation may be 6 _upgradeTo ( implementation ) ;
contained in another statement. Nevertheless, we could not find a 7 ( bool success ,) = delegate . delegatecall ( data );
recurrent pattern in the inconsistent behavior of the detection tool. 8 require ( success ) ;
9 }
Delegatecall to untrusted callee (DTU). The analysis of false nega-
tives revealed two interesting aspects. On the one hand, we noticed
that Slither fails to detect simple cases like the one shown in
Figure 5. On the other hand, we noticed that most delegatecall
5.2 Mutations and Side Effects
functions are used in the constructor, which is invoked only at
deployment time. Therefore, it is impossible to inject an instance of This section analyzes the side effects that mutation testing could
the vulnerability into the constructor that is exploitable. In addition, have on smart contracts. We observed that injecting vulnerabilities
the delegatecall is often rewritten using the assembly to create through mutational operators often introduces side effects, like
a custom function to delegate. Although the mutational operator code smells. We relied on the Slither official documentation5 to
showed a poor injection rate (0.23%), the impact of this vulnerability 5 [Link]
Automated Vulnerability Injection in Solidity Smart Contracts:
A Mutation-Based Approach for Benchmark Development EASE 2025, 17–20 June, 2025, Istanbul, Türkiye

understand the functionality of its detectors and the labels they use not include recognized features. This inconsistency highlights a
for vulnerabilities. significant issue: regardless of the functionality implemented, an in-
correct modifier implementation should always be detected, which
Unchecked low-level call return value (UC) and Unchecked send
is especially critical when the modifier is responsible for ensuring
(US). Both vulnerabilities are injected using the same approach.
that the function invoker is the contract owner.
The differences are in the conditions used to inject them, but the
type of mutation is quite similar. Indeed, the side effects that affect Unused return (UR). We noted that the mutation operator intro-
the injection of these vulnerabilities are the same and occur in duces other vulnerabilities or code smells alongside the injected
75% of the cases. A deprecated-statement occurs when outdated vulnerabilities in 67% of the cases. For example, in some cases,
constructs are used in a contract, e.g., throw instead of revert. The the operator splits a single statement containing declaration and
two mutation operators, UC and US, remove checks on the return initialization into two separate statements, one for declaring the
value of a function. When these checks are implemented using an variable and one for initializing it. The initialization is then made
if statement, the true branch typically reverts the execution of the vulnerable if the return value of a function is used to initialize the
function using a throw construct. By eliminating the check on the variable. The return value is not assigned to the variable, thus cre-
return value, the condition to trigger the throw is also removed, ating an unused return and leaving the variable uninitialized. The
resulting in the removal of the throw from the mutated code. Since side effect of this mutation is the creation or removal of some code
throw is deprecated, its removal eliminates using a deprecated smells. The mutation has introduced uninitialized-state (27,890),
statement in 3,772 cases. initialized-local (97,932), or constable-states (28,352). While not ex-
Analyzing cases (3,697) where the side effect is the missing-zero- pected, these side effects are understandable, given the type of
check reveals an interesting finding. This issue arises when no mutation introduced. Variables uninitialized due to the mutation
validation is performed to ensure that an address, either used as an introduce uninitialized-state and uninitialized-local, depending on
argument or on which a function is invoked, is not the zero address the scope of the uninitialized variable. In cases where the variable
(address(0)). The zero address is often used as a burn address for affected by the mutation undergoes no other changes in the code,
tokens. When the mutation operator modifies the condition by mov- we have a constable-states, a variable not declared constant.
ing it outside an if statement, Slither identifies the presence of a Interesting side effects are those that remove some vulnerabilities
missing zero check. Although the statement remains unchanged, or code smells present before the mutation and that were removed
its move outside the if condition allows Slither to detect an issue by the mutation. For example, divide-before-multiply (-3,913) is
that should be detected regardless of its position in the code. resolved when the return value of multiplication is not assigned to
the variable in which the result of the previous division was saved.
Authorization via [Link] (TX). The mutation operator intro-
Another is incorrect-equality (-3,851), which occurs when a variable
duces some unexpected behavior in 70% of the cases. When a func-
to which the contract balance is assigned is used in a strict equality.
tion performs critical mathematical operations on the contract bal-
Removing the contract balance assignment from the variable also
ance, these operations must be signaled by throwing an event. Func-
removes the vulnerability. Then we have reentrancy-benign (-5,340),
tions that perform mathematical operations on the contract balance
reentrancy-no-eth (-3,191), reentrancy-eth (-830), and reentrancy-
and do not emit an event are labeled by Slither as events-math.
unlimited-gas (-818), all of which are removed from the mutation
When the TX operator mutates them, Slither stops labeling them
because by not assigning the return value to a variable involved
vulnerable in 2,581 cases. The mutation does not involve the mathe-
in reentrancy, it does not change state and does not create the
matical operations in the contract; therefore, they should continue
preconditions for reentrancy. Additionally, controlled-array-length
to be labeled as events-math. One possible explanation could be the
was added and removed depending on the case. It was introduced
absence of the [Link] variable, which seems necessary for a
in 371 mutants but removed in 790. It was eliminated when the
math operation to be signaled by an event.
array index was not initialized using the return value of a function
A similar case involves events-access, emitting an event when-
and introduced when the index remained uninitialized.
ever the [Link] variable is used to change the owner of the
Interestingly, some vulnerabilities or code smells emerged, al-
contract. In 2,398 cases, it is understandable to eliminate this vul-
though not directly connected to the mutation. For example, external-
nerability when [Link] is replaced with [Link].
function or timestamp were introduced by the UR operator but ap-
The last side effect, called by Slither suicidal, occurs 1,058
peared in different lines of code than those affected by the mutation.
times and concerns using the selfdestruct function, which should
The mutated statement and the statement affected by the newly
be restricted to the contract owner. When the mutation replaces
emerged vulnerability shared no common functions or variables.
[Link] with [Link] in the modifier used to give access to
We hypothesize that these instances represent false positives, po-
selfdestruct, Slither correctly detects the misuse of selfdestruct.
tentially caused by conflicts within the Slither execution flow or
The detector applies the appropriate control, discussed in the False
interactions between its detectors.
Negative Analysis section—specifically, verifying the contract owner
using a modifier. However, the detection behavior appears incon- Multiple calls in a loop (CL). The CL operator introduces side
sistent depending on the functionality implemented within the effects that are strictly related to the vulnerability injected in 30% of
function. For instance, if the function includes the invocation of the cases. It introduces msg-value-loop (6617), costly-loop (451), and
a well-known operation, such as selfdestruct, Slither verifies cyclomatic-complexity (126). The first case occurs when [Link]
access to the function by analyzing the associated modifier. Con- is used in mathematical operations inside a loop. The second occurs
versely, Slither fails to check the modifier if the function does when we use costly operations inside a loop that might waste gas, so
EASE 2025, 17–20 June, 2025, Istanbul, Türkiye Iuliano et al.

optimizations are justified. The last case occurs when the cyclomatic not consider. We restricted the false negative analysis to observ-
complexity is higher than 10. In this case, all side effects are closely ing a statistically significant sample from which we extracted the
related to the mutation and, in some cases, are unavoidable. findings reported in the paper.
A second threat concerns using SmartBugs [12], which accel-
Delegatecall to untrusted callee. The DTU operator introduces erates the experiments and simplifies the analysis of the results.
missing-zero-check and naming-convention. The mutation intro- We followed its official documentation, which lists some issues and
duces a new address called delegate and a new function to set the ad- discusses how to mitigate them. Nevertheless, we acknowledge that
dress called setDelegate. The setter function does not check that the our study could have been threatened by relying on this framework.
address is not the zero address (address(0)). The naming-convention Another threat to construct validity relates to MuSe implemen-
is a warning that refers to the address argument of setDelegate; it is tation. Our tool is based upon SuMo [2], a widely recognized and
not in mixedCase violating the Style Guide of Solidity6 . tested mutation testing tool that has already been extended for re-
¤ Discussion Summary. Slither exhibits inconsistent behav- gression mutation testing [4]. However, by relying on SuMo, MuSe
ior and misses vulnerabilities in specific scenarios, suggesting may inherit its defects, potentially impacting the performance of
potential gaps in its detection algorithms. Furthermore, its abil- MuSe. To mitigate the potential issue, we manually validated our
ity to detect some warnings depends on the presence of specific extensions using a statistically significant set of smart contracts
variables, even when these variables are not directly related to composed of a random subset of the smartbugs-wild dataset.
the core logic of the vulnerability. Lastly, choosing the dataset for our experiments and validation
introduces a potential threat to external and construct validity. We
selected the smartbugs-wild dataset [15], one of the most widely
6 Threats To Validity used datasets in the literature. The dataset is recognized for its
This section describes the potential threats to validity, including considerable size and frequent use in empirical investigations, rein-
construct, internal, external, and conclusion validity. forcing its relevance and reliability for our study.

Internal Validity. The vulnerability injection approach threatens

the validity of our study. The initial version of our mutation opera- 7 Conclusion
tors exhibits side effects, which can impact the results. As discussed
in Section 5, some side effects are understandable and legitimate, This paper explores a mutation-based approach to inject known
others are unavoidable, and some can be mitigated by refining the vulnerabilities into smart contracts to generate new benchmarks to
mutation operators. evaluate vulnerability detection tools. We proposed MuSe, a muta-
Another threat concerns the relationship between the injected tion seeding tool that implements mutational operators that identify
vulnerability and its side effects. Sometimes, the code smells that the appropriate pattern in which to inject the mutation. MuSe is ca-
emerged as side effects were not attributable to the injected vulner- pable of injecting six vulnerabilities. An injection rate characterizes
ability. Mutations could introduce code smells independently of the each vulnerability since not all smart contracts have the conditions
injected vulnerability. We have discussed all observed side effects to be mutated and thus be vulnerable to a problem. MuSe has been
and explained their underlying causes, aiming to increase aware- validated and is easily extended by adding new mutational opera-
ness about their proper usage and highlight areas for improvement tors. Slither, a static analyzer, analyzed smart contracts generated
in the mutation operators. by mutation. The results showed gaps in Slither detectors, show-
ing the current limitations of static analyzers and leaving room for
External Validity. We relied on Slither [16] as the static analysis improvement. Our study highlights that using new benchmarks
tool to detect injected vulnerabilities, which may limit the general- generated through a mutation-based approach can improve the
izability of our findings regarding static analysis tools. However, validation of static analysis tools.
Slither is one of the most widely used tools in smart contract analy- As part of our future work, we aim to enhance MuSe by intro-
sis, recognized for its fast execution time and popularity in academic ducing additional mutation operators and refining existing ones
and industrial settings. It is open-source, actively maintained, and to reduce unintended behavior. Our ultimate goal is to develop a
readily accessible, making it ideal for this study. Importantly, the fully automated tool capable of generating vulnerable smart con-
experiment could be easily extended to evaluate additional tools. tracts starting from an initial set of smart contracts. We plan to
Another external threat to validity is the focus on only six vulner- address the gaps identified by Bobadilla et al. [6] by creating new
abilities. However, these vulnerabilities were selected because they datasets. The current literature offers limited labeled benchmarks,
are among the most frequently discussed in the literature [38] and most of which consist of smart contracts written in older versions
represent common security issues in smart contracts. In addition, of Solidity. Using our mutation seeding tool, we intend to inject
the tool can be easily extended by implementing new mutation vulnerabilities into audited and updated smart contracts to produce
operators to inject new vulnerabilities. new, labeled benchmarks and fill these critical gaps.
Construct Validity. A threat to construct validity concerns the
analysis of false negatives. An exhaustive analysis of all false neg-
atives would take too much time and might reveal cases we did Acknowledgments
Finanziato dall’Unione Europea - Next Generation EU, Missione 4
6 [Link] Componente 1 CUP D53D23008400006.
Automated Vulnerability Injection in Solidity Smart Contracts:
A Mutation-Based Approach for Benchmark Development EASE 2025, 17–20 June, 2025, Istanbul, Türkiye

References [23] Satpal Singh Kushwaha, Sandeep Joshi, Dilbag Singh, Manjit Kaur, and Heung-No
[1] Nicola Atzei, Massimo Bartoletti, and Tiziana Cimoli. 2017. A survey of attacks Lee. 2022. Systematic Review of Security Vulnerabilities in Ethereum Blockchain
on ethereum smart contracts (sok). In Principles of Security and Trust: 6th Interna- Smart Contract. IEEE Access 10 (2022), 6605–6621. doi:10.1109/ACCESS.2021.
tional Conference, POST 2017, Held as Part of the European Joint Conferences on 3140091
Theory and Practice of Software, ETAPS 2017, Uppsala, Sweden, April 22-29, 2017, [24] Satpal Singh Kushwaha, Sandeep Joshi, Dilbag Singh, Manjit Kaur, and Heung-No
Proceedings 6. Springer, 164–186. Lee. 2022. Systematic review of security vulnerabilities in ethereum blockchain
[2] Morena Barboni, Andrea Morichetta, and Andrea Polini. 2021. SuMo: A Mutation smart contract. IEEE Access 10 (2022), 6605–6621.
Testing Strategy for Solidity Smart Contracts. In 2021 IEEE/ACM International [25] Chao Liu, Han Liu, Zhao Cao, Zhong Chen, Bangdao Chen, and Bill Roscoe.
Conference on Automation of Software Test (AST). 50–59. doi:10.1109/AST52587. 2018. Reguard: finding reentrancy bugs in smart contracts. In Proceedings of the
2021.00014 40th International Conference on Software Engineering: Companion Proceeedings.
[3] Morena Barboni, Andrea Morichetta, and Andrea Polini. 2022. SuMo: A mutation 65–68.
testing approach and tool for the Ethereum blockchain. Journal of Systems and [26] Loi Luu, Duc-Hiep Chu, Hrishi Olickel, Prateek Saxena, and Aquinas Hobor.
Software 193 (2022), 111445. doi:10.1016/[Link].2022.111445 2016. Making smart contracts smarter. In Proceedings of the 2016 ACM SIGSAC
[4] Morena Barboni, Andrea Morichetta, Andrea Polini, and Francesco Casoni. 2024. conference on computer and communications security. 254–269.
ReSuMo: a regression strategy and tool for mutation testing of solidity smart [27] Muhammad Izhar Mehar, Charles Louis Shier, Alana Giambattista, Elgar Gong,
contracts. Software Quality Journal 32, 1 (2024), 225–253. Gabrielle Fletcher, Ryan Sanayhie, Henry M Kim, and Marek Laskowski. 2019.
[5] Kent Beck. 2022. Test driven development: By example. Addison-Wesley Profes- Understanding a revolutionary and flawed grand experiment in blockchain: the
sional. DAO attack. Journal of Cases on Information Technology (JCIT) 21, 1 (2019),
[6] Sofia Bobadilla, Monica Jin, and Martin Monperrus. 2025. Do Automated Fixes 19–32.
Truly Mitigate Smart Contract Exploits? arXiv preprint arXiv:2501.04600 (2025). [28] Alexander Mense and Markus Flatscher. 2018. Security vulnerabilities in
[7] Vitalik Buterin et al. 2014. Ethereum white paper: a next generation smart ethereum smart contracts. In Proceedings of the 20th international conference
contract & decentralized application platform. First version 53 (2014). on information integration and web-based applications & services. 375–380.
[8] Patrick Chapman, Dianxiang Xu, Lin Deng, and Yin Xiong. 2019. Deviant: A [29] Mike Papadakis, Marinos Kintis, Jie Zhang, Yue Jia, Yves Le Traon, and Mark
Mutation Testing Tool for Solidity Smart Contracts. In 2019 IEEE International Harman. 2019. Chapter Six - Mutation Testing Advances: An Analysis and Survey.
Conference on Blockchain (Blockchain). 319–324. doi:10.1109/Blockchain.2019. Advances in Computers, Vol. 112. Elsevier, 275–378. doi:10.1016/[Link].2018.
00050 03.015
[9] Ting Chen, Yuxiao Zhu, Zihao Li, Jiachi Chen, Xiaoqi Li, Xiapu Luo, Xiaodong [30] Reza M Parizi, Ali Dehghantanha, Kim-Kwang Raymond Choo, and Amritraj
Lin, and Xiaosong Zhange. 2018. Understanding Ethereum via Graph Analysis. In Singh. 2018. Empirical vulnerability analysis of automated smart contracts
IEEE INFOCOM 2018 - IEEE Conference on Computer Communications. 1484–1492. security testing on blockchains. arXiv preprint arXiv:1809.02702 (2018).
doi:10.1109/INFOCOM.2018.8486401 [31] Heidelinde Rameder, Monika Di Angelo, and Gernot Salzer. 2022. Review of
[10] Hanting Chu, Pengcheng Zhang, Hai Dong, Yan Xiao, and Shunhui Ji. 2024. SGDL: automated vulnerability analysis of smart contracts on Ethereum. Frontiers in
Smart contract vulnerability generation via deep learning. Journal of Software: Blockchain 5 (2022), 814977.
Evolution and Process 36, 12 (2024), e2712. [32] Meng Ren, Zijing Yin, Fuchen Ma, Zhenyang Xu, Yu Jiang, Chengnian Sun,
[11] Etienne Daspe, Mathis Durand, Julien Hatin, and Salma Bradai. 2024. Bench- Huizhong Li, and Yan Cai. 2021. Empirical evaluation of smart contract testing:
marking Large Language Models for Ethereum Smart Contract Development. What is the best choice?. In Proceedings of the 30th ACM SIGSOFT international
1–4. doi:10.1109/BRAINS63024.2024.10732686 symposium on software testing and analysis. 566–579.
[12] Monika di Angelo, Thomas Durieux, João F. Ferreira, and Gernot Salzer. 2023. [33] Noama Fatima Samreen and Manar H Alalfi. 2021. Smartscan: an approach
SmartBugs 2.0: An Execution Framework for Weakness Detection in Ethereum to detect denial of service vulnerability in ethereum smart contracts. In 2021
Smart Contracts. In 38th IEEE/ACM International Conference on Automated IEEE/ACM 4th International Workshop on Emerging Trends in Software Engineering
Software Engineering (ASE). IEEE Computer Society, 2102–2105. doi:10.1109/ for Blockchain (WETSEB). IEEE, 17–26.
ASE56229.2023.00060 [34] Sarwar Sayeed, Hector Marco-Gisbert, and Tom Caira. 2020. Smart contract:
[13] Monika Di Angelo and Gernot Salzer. 2019. A survey of tools for analyzing Attacks and protections. Ieee Access 8 (2020), 24416–24427.
ethereum smart contracts. In 2019 IEEE international conference on decentralized [35] Fabian Schär. 2021. Decentralized finance: On blockchain-and smart contract-
applications and infrastructures (DAPPCON). IEEE, 69–78. based financial markets. FRB of St. Louis Review (2021).
[14] Ardit Dika and Mariusz Nowostawski. 2018. Security Vulnerabilities in Ethereum [36] Fernando Richter Vidal, Naghmeh Ivaki, and Nuno Laranjeiro. 2024. OpenSCV:
Smart Contracts. In 2018 IEEE International Conference on Internet of Things an open hierarchical taxonomy for smart contract vulnerabilities. Empirical
(iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Software Engineering 29, 4 (2024), 101.
Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData). [37] Xiaoyin Wang, Jiaze Sun, Chunyang Hu, Panpan Yu, Bin Zhang, and Donghai
955–962. doi:10.1109/Cybermatics_2018.2018.00182 Hou. 2022. EtherFuzz: mutation fuzzing smart contracts for TOD vulnerabil-
[15] Thomas Durieux, João F Ferreira, Rui Abreu, and Pedro Cruz. 2020. Empirical ity detection. Wireless Communications and Mobile Computing 2022, 1 (2022),
review of automated analysis tools on 47,587 ethereum smart contracts. In Pro- 1565007.
ceedings of the ACM/IEEE 42nd International conference on software engineering. [38] Oualid Zaazaa and Hanan El Bakkali. 2023. A systematic literature review of
530–541. undiscovered vulnerabilities and tools in smart contract technology. Journal of
[16] Josselin Feist, Gustavo Grieco, and Alex Groce. 2019. Slither: a static analysis Intelligent Systems 32 (09 2023). doi:10.1515/jisys-2023-0038
framework for smart contracts. In 2019 IEEE/ACM 2nd International Workshop on [39] Zibin Zheng, Jianzhong Su, Jiachi Chen, David Lo, Zhijie Zhong, and Mingxi Ye.
Emerging Trends in Software Engineering for Blockchain (WETSEB). IEEE, 8–15. 2024. DAppSCAN: Building Large-Scale Datasets for Smart Contract Weaknesses
[17] Asem Ghaleb and Karthik Pattabiraman. 2020. ‘. In Proceedings of the 29th ACM in DApp Projects. IEEE Trans. Softw. Eng. 50, 6 (June 2024), 1360–1373. doi:10.
SIGSOFT International Symposium on Software Testing and Analysis (Virtual Event, 1109/TSE.2024.3383422
USA) (ISSTA 2020). Association for Computing Machinery, New York, NY, USA, [40] Weiqin Zou, David Lo, Pavneet Singh Kochhar, Xuan-Bach Dinh Le, Xin Xia,
415–427. doi:10.1145/3395363.3397385 Yang Feng, Zhenyu Chen, and Baowen Xu. 2021. Smart Contract Development:
[18] Ákos Hajdu, Naghmeh Ivaki, Imre Kocsis, Attila Klenik, László Gönczy, Nuno Challenges and Opportunities. IEEE Transactions on Software Engineering 47, 10
Laranjeiro, Henrique Madeira, and András Pataricza. 2020. Using Fault Injection (2021), 2084–2106. doi:10.1109/TSE.2019.2942301
to Assess Blockchain Systems in Presence of Faulty Smart Contracts. IEEE Access
8 (2020), 190760–190783. doi:10.1109/ACCESS.2020.3032239
[19] Gerardo Iuliano and Dario Di Nucci. 2024. Smart Contract Vulnerabil-
ities, Tools, and Benchmarks: An Updated Systematic Literature Review.
arXiv:2412.01719 [[Link]] [Link]
[20] Y. Ivanova and A. Khritankov. 2020. RegularMutator: A Mutation Testing Tool
for Solidity Smart Contracts. Procedia Computer Science 178 (2020), 75–83.
doi:10.1016/[Link].2020.11.009 9th International Young Scientists Conference in
Computational Science, YSC2020, 05-12 September 2020.
[21] Yue Jia and Mark Harman. 2011. An Analysis and Survey of the Development
of Mutation Testing. IEEE Trans. Softw. Eng. 37, 5 (Sept. 2011), 649–678. doi:10.
1109/TSE.2010.62
[22] Bo Jiang, Ye Liu, and Wing Kwong Chan. 2018. Contractfuzzer: Fuzzing smart
contracts for vulnerability detection. In Proceedings of the 33rd ACM/IEEE inter-
national conference on automated software engineering. 259–269.