Received 15 November 2024, accepted 19 December 2024, date of publication 24 December 2024, date of current version 6 January 2025.

Digital Object Identifier 10.1109/ACCESS.2024.3522094

Blending Static and Dynamic Analysis for Web

Application Vulnerability Detection:
Methodology and Case Study
PAULO NUNES 1, JOSÉ FONSECA 1, AND MARCO VIEIRA 2, (Member, IEEE)
1 Polytechnic of Guarda, University of Coimbra, CISUC, 3004-531 Coimbra, Portugal
3 University of North Carolina at Charlotte, Charlotte, NC 28223, USA

Corresponding author: Paulo Nunes (pnunes@[Link])

ABSTRACT Static Analysis (SA) and Dynamic Analysis (DA) are complementary techniques for searching
web application vulnerabilities. Typically, SA detects more vulnerabilities but reports a higher number
of false positives, whereas DA finds less but with better precision. In this paper, we blend SA and DA
to simultaneously improve the detection and decrease the false alarms. Our approach starts with SA to
identify an initial set of potential vulnerabilities. Then, the target application is executed to obtain specific
runtime information. These data are used to automatically configure the DA, improving its ability to confirm
if the vulnerabilities reported by the SA are indeed exploitable. We evaluated the proposed approach
using 49 WordPress plugins with more than 450 SQLi vulnerabilities. Our approach was able to confirm
either as a vulnerability or a false alarm 76.7% of the results reported by the SA, decreasing tremendously
the usual need for manual work, which is a huge improvement for security practitioners.

INDEX TERMS Static analysis, dynamic analysis, vulnerability detection, execution traces, SQLi, blend
analysis.

I. INTRODUCTION is to first conduct a Static Analysis (SA) and later on to run

The number of security vulnerabilities in web applications Dynamic Analysis (DA) tests [7], [8]. The ability to analyze
has increased along with the tremendous growth of web an application both statically and dynamically is of utmost
applications in the last two decades [1]. In spite of all importance, as some vulnerabilities are better detected with
the knowledge available about the security risks involved, SA, while others with DA. In general, SA is able to cover
this situation seems to be difficult to improve [2], [3], [4]. 100% of the source code, but it produces many False Positive
An Enterprise Strategy Group research in 2020, reports some (FP) cases [4]. On the other hand, DA is usually only able to
of the reasons why companies release vulnerable code to cover part of the code, but it is very precise when it signals
production: 54% to meet a critical deadline, 49% consider a vulnerability [9], [10], [4]. Testing with both SA and DA
the vulnerabilities as a low risk, and 45% discover the issues yields the most comprehensive testing and it is considered
too late in the Software Development Life Cycle (SDLC). a must have by secure SDLCs. They are typically applied
Web application vulnerabilities were the cause of 39% of separately and in different stages of the development process:
data breaches in 2022, according to The 2023 Verizon Data SA is used earlier when compiling the software modules
Breach Investigations Report [5]. This confirms the crucial and DA later when the modules are already developed and
importance of having a solid CI/CD pipeline where security integrated with each other.
is totally integrated as early as possible in the SDLC [6]. The complementary advantages of SA and DA have led
One of the most effective ways to lower the number of researchers into combining them to achieve the best of both
vulnerabilities present in the source code during the SDLC, worlds (e.g., [9], [11], [12], [13], [14]). One direction to
increase the precision of the SA is to confirm the reported
The associate editor coordinating the review of this manuscript and vulnerabilities by using DA [9]. Some approaches aim at
approving it for publication was Joao Bernardo Ferreira Sequeiros . preventing online attacks, thus requiring runtime components
2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License.
VOLUME 13, 2025 For more information, see [Link] 3139
 P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

of the application while it is in production [15]. Other the vulnerabilities so they can fix them, while spending the
approaches require the generation of specific test inputs least amount of resources dealing with false alarms.
to detect vulnerabilities [16]. However, the generation of The proposed solution improves the effectiveness of state-
effective test inputs is a hard task and there are research of-the-art of DA tools by providing useful data from the SA
works dedicated to address this problem [17], [18]. Despite and the crawling procedure to exploit the Sensitive Sink (SS).
the fact that SA provides the origin of the vulnerability (i.e., SA provides the location (Lines of Code (LOC)) of the vul-
vulnerable input variable, file name, line), it is generally nerable code including the Entry Points (EPs) and vulnerable
blinded towards the values of other relevant input variables variables to be tested by the DA. It also provides useful data
that are going to define the path constraints that must for guiding the crawling process. For example, knowing that
be satisfied so that the vulnerable input variable is able a vulnerability is located in an INSERT Structured Query
to travel through the code and succeeds in activating the Language (SQL) statement, we can navigate right away to
vulnerability [18]. Therefore, approaches based only on the the web interface performing this operation. The crawling
results of SA to feed the DA have the limitation of not process provides the Uniform Resource Locator (URL),
providing enough data to demonstrate the exploitability of parameters and concrete values triggering the vulnerability.
all the vulnerabilities discovered. This paper proposes a Implicitly, the crawling process also provides the coverage of
methodology to effectively increase the number of reported the SA results covered during the process. It is important to
True Positives (TPs) while reducing the number of FPs, emphasize that it is very difficult to obtain these data, for the
when searching for vulnerabilities using SA and DA. DA tools alone. In fact, this is a reason for the low coverage of
In practice, we combine SA and DA with the inclusion of DA tools. These are aspects that require more detailed studies,
a runtime procedure (crawling) in between, in a way that such as the one presented here [22].
the information provided to the DA by the SA and the The main contributions of the paper are the following:
crawling allows the creation of tailored attack vectors that • A generic methodology that combines SA, a crawling
can effectively be used to increase the number of confirmed procedure, and DA into a number of steps, that can be
TPs and, at the same time, reduce the overall number fully automated, in order to obtain a set of vulnerabilities
of FPs. In short, our methodology starts by performing a that are confirmed to be exploitable.
SA to produce a list of candidate vulnerabilities. Next, the • A new technique for gathering useful runtime data
application is executed, automatically stopping only when during the process of crawling applications. The tech-
the code where those candidate vulnerabilities are located nique correlates static analysis data with runtime data to
is run. A set of specific inputs and configuration options generate the configuration data required to an optimized
is automatically generated from the results of the SA and exploitation of the vulnerability using dynamic analysis.
the runtime information collected, which will guide the The technique also reports the coverage of vulnerabili-
DA in the process of exploiting each vulnerability reported ties triggered during the crawling process.
by the SA. Unlike other approaches, our methodology • An example of the instantiation of the methodology to
has the ability to automatically generate the necessary data demonstrate its feasibility, using a set of SQL Injection
to feed the DA, so it is able to go through the execution (SQLi) vulnerabilities in 49 WordPress plugins.
path where the vulnerability is located and trigger it, The paper is organized as follows. Section II discusses the
therefore increasing the number of vulnerabilities that can be technical background and related work. Section III describes
confirmed by the DA. the proposed methodology, while Section IV describes the
For demonstration purposes, our approach was experi- empirical study conducted to evaluate it. Section V presents
mentally evaluated with SQLi vulnerabilities in 49 freely and discusses the results. Section VI concludes the paper.
available vulnerable WordPress plugins developed using
PHP. PHP is by far the most popular server-side programming II. BACKGROUND AND RELATED WORK
language. It powers 75.7% of all web sites [19]. WordPress A web application attack can be considered as a malicious
is the most popular Content Management Systems (CMS) external interaction exploiting a security vulnerability to
nowadays, being used by 43.6% of all websites (as of attempt an intrusion that may cause an error and possibly
October 31, 2024), and has a CMS market share of subsequent failures of the system [23]. An attack vector is
62.4% [20]. Also, the WordPress environment has more a set of input parameter values that allows the attack to
vulnerabilities than any other CMS. travel in the internal web application path allowing the deliver
To support the SA, we used a diverse set of SA tools (that of a payload triggering a specific vulnerability. A security
overall improve the detection capability of each individual vulnerability is a weakness that may be exploited to cause
tool [21]). We used the well-known SQLMap as the DA harm, but its presence does not cause harm by itself [24].
tool. Results show that our approach is able to confirm This weakness is usually created by a deficient input data
480% more TPs and 700% more FPs than when directly protection mechanism of the application. Data protection
feeding the results of the SA to the DA. Our approach mechanisms can be implemented as validation (rejects data
provides a huge improvement for developers and security that does not meet a predefined pattern), sanitization (cleans
practitioners that need to use the best possible data to identify data of unwanted characters), or escaping (delimits special

3140 VOLUME 13, 2025

P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

characters with control characters in order to remove their the application. In practice, malicious patterns (special input
unwanted meaning) [25]. When a vulnerability is exploited, test cases) are used to simulate the web application user
the attack uses input variables to trigger the vulnerability in interaction. When an error or specific behavior are observed,
the SS. A SS is the piece of code, which can be a function call it is assumed that a possible vulnerability was triggered [33].
for example, that exposes private data to external systems or However, many times it is difficult to analyze the malicious
executes a malicious operation. An example of a SS function effects of the interaction because they may not be visible
is the PHP mysql_query function, which executes a SQL from the interface, which is the only access that the DA has
query and returns the results [26]. to the application. For example, a malicious update (or a
There is a SQLi vulnerability when untrusted data is delete) operation in a database may be executed silently, not
allowed to flow from an EP with inadequate protection providing any evidence to the user interface.
and is used for the construction of SQL queries [27]. The SA and DA have complementary advantages, and this has
attack occurs when an adversary maliciously injects SQL led researchers to devise combinations of static and dynamic
commands, via an unprotected input field, in a way that analysis.
would alter the SQL statement executed by the database Halfond and Orso [11] proposed a technique (AMNESIA)
server. Attackers use SQLi to illegally get access to the that uses a model-based approach to detect illegal queries
application’s database, allowing them to harvest and possibly before they get executed on the database. AMNESIA
change sensitive information and, therefore, gain control of performs a DA to extract the structure of legitimate SQL
the application for their personal benefit [28]. queries to build a model for each vulnerability. Using runtime
There are many techniques to search for vulnerabilities monitoring, it compares the dynamically generated queries
in web applications, but SA and DA are two of the most with the model to check if the query can be executed or
widely used and are also complementary in several important not. This approach reported no FPs, however, the model
aspects [29], [30]. SA performs the analysis by looking at accuracy is bounded by the flow-insensitive behavior of the
the source code, while DA requires executing the program. SA performed to extract the vulnerability code.
They both have a dominant role in the technology used Su and Wassermann [12] proposed an algorithm
to uncover software bugs, namely security vulnerabilities (SQLCheck) for preventing SQLi attacks based on
like SQLi, during the software development lifecycle [29], context-free grammars and compiler parsing techniques.
[30]. SA provides a list of vulnerable lines of code and It parses the syntax of the SQL query to gather the types of the
allows understanding the conditions that should be satisfied to arguments. It detects malicious code when the user input does
trigger the vulnerability. DA reports a list of vulnerabilities, not match with the registered data type. This tool produced
along with the vulnerable program input values that trigger no FPs, no False Negatives (FNs), had low runtime overhead,
those vulnerabilities. and could be applied to web applications written in different
Both SA and DA are very time consuming (and DA is not languages. Its main disadvantage is the need to rewrite the
easily scalable), so they are usually performed with the help SQL statements that were dynamically generated.
of automated tools. These tools help finding vulnerabilities The approach proposed by Balzarotti et al. [9] first uses
much quicker than manually and are of utmost importance SA to identify built-in and custom protection routines. Then,
for developers. However, they have several limitations, to reduce the number of FPs, it uses DA to confirm the
like incomplete vulnerability detection, high false alarm existence of potential vulnerabilities in these routines. They
rate, and expensive licenses (research and practice show simulate the effect of the routine on the input by executing
that the effectiveness of existing tools is far from being the code with different test inputs containing various types of
satisfactory [31], [32]). attack strings. This approach is limited to applications using
SA tools are relatively fast and can be applied early in such protection routines, which misses the vulnerabilities that
the SDLC for detecting security vulnerabilities as soon as are due to non-existent data protection.
possible, even before the code is compiled. However, SA has Lee et al. [14] combined both SA and DA for detecting
some limitations regarding the precision of the results, when SQLi attacks. They started by using SA to obtain the
compared to other techniques, like DA. In fact, it suffers from structure of the query, after removing the value of attributes
the conceptual limitation of undecidability that does not affect involved in SQL queries (e.g., values enclosed within
DA [9]. However, it is able to work in parts of the code that quotes, or values followed by the = symbol). Afterwards,
DA may never reach, because they are never called or are at runtime, the dynamically generated query is captured.
called from a different entry point than the one being used by Attacks are detected if the structure and the grammar of the
the DA. SQL statements are changed. The results showed that the
DA, such as penetration testing, is based on the application proposed method is very effective and simpler than other
analysis from the user-side perspective, thus its effectiveness methods, such as AMNESIA and SQLCheck, which cannot
depends on the quality of the test cases used. As it assumes detect stored procedure attacks. The primary limitation
that the source code is not available, DA is generally of this approach is that it can only detect known SQL
independent from the programming language used to develop models.

VOLUME 13, 2025 3141

 P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

Alhuzali et al. [34] proposed an approach combining DA configuration of the options that the DA tool should use to be
that is guided by SA techniques in order to automatically able to create the payloads that effectively exploit the target
identify vulnerabilities and build working exploits taking vulnerability (see Fig. 1).
into account the dynamic features and the navigational Obtaining the data needed for the setup of the DA
complexities of web applications (e.g., dynamically gener- tool is a challenging task. For example, the vulnerable
ated forms and links that may drive the navigation of the web application file stored in the web server may not be
application to vulnerable SSs). The approach is implemented directly callable from a simple URL, since many times it is
in a tool called NAVEX. The evaluation results show that only accessible from within a specific context of the main
NAVEX outperforms other approaches and scales to very application (as in the case of a WordPress plugin file). In order
large applications and to multiple classes of vulnerabilities. to do this automatically, we have to find a way to learn the
Experimental results showed that NAVEX was able to process (sequence of steps and input values) that the main
generate concrete exploits for 68% of the SQLi exploitable application uses to execute the vulnerable page and be able to
sinks. The enhanced SA reduced FPs by 87% on average. repeat it later on with different payloads. SA tools typically
The inclusion of client-side code analysis for building the identify the vulnerable source code variables, but for the DA
navigation graph enhanced the precision of exploit generation tools we need the input Hypertext Transfer Protocol (HTTP)
by 54% on average. However, the dataset used to evaluate parameters and their specific values that will be transferred
the tool is very old (the applications vary between 2004 and to source code input variables able to trigger the execution
2008 and one is from 2012) and are not characterized in path that reaches the SS where the vulnerability is located.
terms of vulnerabilities (i.e., the number of vulnerabilities in To help in this process, the SA tools need to be configured
the applications is unknown). Despite the ability of the tool in order to provide not only the information about the SS,
for automatic vulnerability detection and exploit generation, but also the input variable that is the entry point to exploit
its vulnerability detection coverage was not evaluated. This the vulnerability. Based on how the vulnerability manifests
would allow assessing the ability of the SA stage to avoid itself, it may be possible to optimize the payloads needed
FPs without missing vulnerabilities. Compared to NAVEX, to exploit it. For example, a SQLi vulnerability located on
our approach can benefit from the strengths of several well a SELECT statement is likely to be exploited differently than
established SA, DA and crawling techniques. For example, if it is located on an UPDATE statement.
our approach can use the results of several diverse SA tools To address the problems above, we propose a methodology
as a way to increase the vulnerability detection coverage, for vulnerability detection, which has six stages, as repre-
while NAVEX is limited to its own SA technique that only sented in Fig. 1 and discussed next:
models 35 PHP built-in functions. The attack dictionary used 1) Obtaining Static Analysis Data: The first stage consists
by NAVEX for concrete exploit generation only contains of collecting the web application vulnerability detection data
45 attack patterns (attack vectors) for the three classes using SA. The SA can be done manually, which is neither
of vulnerabilities that NAVEX detect. Although it can be feasible for most projects nor scalable, so this is usually done
expanded, our approach already takes advantage of DA tools using a tool (or a set of tools to improve performance), like
(e.g., SQLMap) that uses thousands of attack vectors with what developers and security practitioners so often do.
several variants when exploiting vulnerabilities. 2) Gathering runtime Information: This stage aims at
collecting relevant information about the execution of the
III. BLENDED STATIC AND DYNAMIC ANALYSIS application. The application can be interacted manually or
This section describes our generic approach for blending using an automated tool, like the Acunetix Web Vulnerability
static and dynamic analysis in order to build a vulnerability Scanner built-in crawler, while executing all the relevant
detection mechanism for complex and dynamic web appli- actions (e.g., view, insert and delete data). Either way, the
cations that features a high detection rate with reduced false entire interaction is recorded both from within the web server
positives. It is based on the high vulnerability detection rate runtime execution trace files (e.g., using the XDebug tool,
that SA is able to provide and on the use of DA to confirm if the target is a PHP application) and from the HTTP
if the vulnerabilities reported are indeed true positives. The interaction (e.g., OWASP Zed Attack Proxy (ZAP)).
SA can be automated by means of tools (like RIPS [35] or 3) Mapping HTTP Requests With Trace Files: Since the
WAP [36]), and the DA by existing automatic penetration data in the trace files and in the HTTP interactions were
testing tools, like SQLMap (for SQLi) or Xsser (for Cross- obtained independently, using different tools, they are not
Site Scripting (XSS)). In practice, for each vulnerability synchronized. The objective of this stage is to map the HTTP
reported by the SA tool (we refer to it as a single tool, but requests with the respective web server files that belong to
in practice a set of tools can be used, as we will discuss the same user interaction provided by the trace files. The
later on), we need to build a custom configuration setup for end result are pairs of HTTP requests and the web files they
the execution of the DA tool, containing the URL of the executed. This allows us to know how to navigate from the
vulnerable web page (that may not be directly accessible entry point of the web application to the vulnerable page.
via a URL link), the input parameters and their values, Since the HTTP request has the name and value of the
the identification of the vulnerable input parameter and the input variables, and the trace file has the name of the web

3142 VOLUME 13, 2025

P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

FIGURE 1. Methodology overview.

server file executed, this mapping also allows us to obtain, different tools tend to report distinct sets of vulnerabilities,
for each vulnerable file, the parameters and their concrete with some overlap [38]. Therefore, combining multiple SA
values, enabling the execution of the code that trigger the tools has the potential benefit of improving the number of
vulnerabilities. vulnerabilities detected when compared with using just a
4) Generating the DA Configuration: This stage aims single SA tool, due to the complementary nature of the
to automatically generate the configuration of the DA tool results. However, this process is also likely to generate more
in order to exploit the vulnerability. From the previous FPs, since it is well-known that using more tools increase
stages, we obtained the URL, the input HTTP parameters both TP and FP [37], [39]. To achieve the right balance of
and their values, and the identification of the vulnerable TPs and FPs, previous works tested different combinations
HTTP parameters. From the SA output, we can parse the of tools, and were able to provide the best solution depending
LOC of the SS in order to find useful attributes, like where on the target scenario [37]. This flexibility, allied to our
each vulnerability is located within the source code file (e.g. methodology that allows confirming precisely which results
a XSS inside an HTML tag should be treated differently are TPs, is a step towards the best of both SA and DA worlds.
than if it is within a Javascript function) and the type of
vulnerability (e.g. a SQLi SELECT statement should be B. GATHERING RUNTIME INFORMATION
exploited differently than an UPDATE statement). With this DA has several limitations related with the difficulty in
knowledge, specific configuration parameters of the DA generating the test cases (i.e., the inputs, such as URLs and
tool can be automatically tweaked in order to improve the parameters) that allow the exploitation of a vulnerability.
likelihood of success and the speed of operation. In fact, the coverage and effectiveness of DA is critically
5) Testing Vulnerability Exploitability: This stage checks affected by the quality of the test cases used. Our approach
if each vulnerability is indeed exploitable. In practice, the to generate good test cases, able to cover nearly 100% of the
DA is performed using the configurations generated in the vulnerable code, is based on real values provided by both the
previous stage. If it succeeds in exploiting the vulnerability, user using the application (e.g., inputs from form fields) and
the DA tool outputs a Proof of Concept (PoC).1 This is a the application itself (e.g., hidden fields and cookies) when
payload that can be manually executed and is the proof that it is being executed. To obtain these data we need to run and
the vulnerability exists and can be exploited. interact with the application, while storing the interaction’s
6) PoC Reporting: The information about the vulnera- inputs and outputs. The interaction with the application can be
bilities obtained from the previous stages is put together, automated using tools that simulate the actions of experienced
including data from the SA and DA tools, in order to have web application penetration testers [40].
a consolidated report. This report presents a list of all the The methodology for gathering this runtime information
vulnerabilities found that are proven to be exploitable and consists of following two steps, represented in Fig. 3.
their respective PoC.
1) CRAWLING THE APPLICATION
A. OBTAINING STATIC ANALYSIS DATA The first step is to gather the runtime information of the
To maximize the quality of the output of the SA, we can use target application. Before, we need to insert a proxy between
the results from a set of diverse SA tools (Figure 2) [37], the user web browser and the network so that it can capture
instead of those from a single tool. In the former case, the SA the HTTP requests of the interaction. We must also store
tools perform the analysis of the source code and the results the server side data. This can be done by means of a web
are combined in a way that the same vulnerability detected server debug feature, like the XDebug file profiler tool for
by several tools is reported only once. Besides the high FP the PHP Stack Trace. For each HTTP request, it creates a
problem, the SA tools tend to miss some vulnerabilities and single trace file with relevant data (e.g., SSs executed and
respective parameters). Next, we need a way to interact with
1 The PoC is an attack against an application that is performed only to the application, which can be done automatically (using an
prove that it can be done, showing how a hacker can take advantage of a automated crawler) or manually. By crawling the application,
vulnerability in the application. we mean executing its functions, filling in the form fields and

VOLUME 13, 2025 3143

 P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

FIGURE 2. Obtaining static analysis data.

clicking on the hyperlinks in order to test the target module used to tailor the payloads to the specific characteristics of the
as thoroughly as possible (including the vulnerable parts). vulnerability and to reduce the number of test cases needed.
While the web application is being interacted, the HTTP
E. TESTING THE VULNERABILITY EXPLOITABILITY
requests and the execution traces are collected and stored
in independent log files. To make it easier to synchronize At this stage, the DA is executed, using the configuration
both capture processes, we should crawl the application created in the previous stage, in order to confirm (in case
sequentially instead of using multiple threads. of detection) the vulnerabilities found by the SA tool. If the
DA cannot exploit the vulnerability it does not guarantee that
2) IDENTIFYING VULNERABLE SSs EXECUTED it is unexploitable, nor a FP reported by the SA, because a
The crawling stops only when all the target SSs identified different attack may succeed. If these failed cases are very
by the SA are reached. This is done automatically, since we important for the developer, they must be verified manually.
already have all the necessary information about the SSs from If a vulnerability is found by the SA tool and the DA can
the SA and the code executed by the runtime information exploit it, then it is proven to be exploitable. On success,
gathered in the previous step. This process assures the ability the DA tool provides a PoC on how the vulnerability can
to obtain, as close as possible, a perfect coverage of all the be exploited. This is usually a URL link that can be used
vulnerabilities reported by the SA. in a web browser or a block of code that can be executed
autonomously in a terminal window. This PoC is the proof
C. MAPPING HTTP REQUESTS WITH TRACE FILES that the vulnerability can indeed be exploited. A PoC example
Since the HTTP requests and the trace files were obtained with a SQLMap usage is shown in Listing 4 (lines 7-8) in
using independent software, they are not readily linked to Section IV-F.
each other. To construct the configuration of the DA tool,
we need to know the URL that contributed to the execution F. PoC REPORTING
of the vulnerable server file, the HTTP input parameters and Reporting results involves communicating the findings of
their values and the identification of the vulnerable input the vulnerability detection in a manner that makes sense
parameter. We also need details about the vulnerable source to the target audience, like system owners and developers.
code, which can be obtained by analyzing the LOCs reported The report displays information about both the vulnerabilities
by the SA tool. The URL obtained from the HTTP requests successfully exploited and the vulnerabilities that were not
must be mapped with the call to the respective web server exploited by the DA. For the vulnerabilities successfully
file present in the trace file. The parameters and their values exploited, it includes the number of requests, the execution
are obtained from the HTTP requests. The identification of time, the attack vectors (i.e., types of attack) and the payloads
the HTTP vulnerable parameter comes from the information (i.e., data needed to replay the attacks). For the vulnerabilities
provided by the SA tool. Since both the trace files and the detected by the SA but that could not be exploited by the DA,
HTTP requests have the timestamp information, and we are it only includes the number of requests and the execution
not using parallelism when crawling, the trace file originated time. It is important to emphasize that we do not consider
from the HTTP request is the first one that was created these cases as FP until there is a manual confirmation by
after the timestamp of that HTTP request. This is why we an expert. Regarding this situation, a real benefit of using
should crawl the application sequentially, which should not our methodology is that this number is greatly reduced
be a problem, since this process is typically done in the web when comparing with typical procedures, as we will see in
application software development environment. Section V.

D. GENERATING THE DA CONFIGURATION IV. EMPIRICAL STUDY AND EXAMPLES

When entering this stage, we already have the mapping This section presents the experimental evaluation of our
between the report of the vulnerable files and variables from approach. It also provides concrete examples on how the
the SA tool with the trace files and the mapping of those with different steps of the approach can be executed in practice.
the HTTP requests. Based on these data we can generate a We instantiate our generic approach for the PHP language,
specific configuration of the DA tool that is able to attack but the concepts behind it are generic and are applicable to
the vulnerability. Depending on the type of vulnerability and other web languages. The components that depend on the
on the injection points, additional parameters may also be programming language are the SA tools and the file profiler.

3144 VOLUME 13, 2025

P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

FIGURE 3. Gathering runtime information.

For both components, there are tools available for the most (TPs+FPs). Therefore, the SA itself drastically reduced the
used server side programming languages such as [Link], number of cases (i.e., SS) to be tested with DA.
Ruby and Java. The question now is how to automatically separate the real
Table 1 lists the WordPress plugins used in our study. vulnerabilities from the FPs, since we know from the manual
These plugins vary widely both in terms of number of LOC annotation that only 462 are TPs that can be exploited and
and number of files. Overall the plugins contain 552,504 140 are FPs. Similar to real world scenarios, our goal here is to
LOC in 1,561 source code files. To have an idea of their demonstrate that the process of confirming the exploitability
relevance, overall these plugins have been downloaded over of the vast majority of the cases can be effectively done using
25 million times and they are used in business, e-commerce, our approach, leaving just a few vulnerabilities that were not
social networking (Google, Youtube), photo and video exploited (thus increasing confidence in the approach).
gallery, registration, admin, advertising, email, bookings,
reservations, events management, newsletter and e-learning. B. GATHERING RUNTIME INFORMATION
The table includes the number of vulnerabilities (TP), false To gather the runtime information of the WordPress plugins,
positives (FP), number sensitive sinks (SS), number of files shown in Table 1 we need to install WordPress, the plugins,
(Files) and number of lines of code (LOC). Note that the setup the testing environment and then run the plugins,
potential number of SS in the plugins are about 8 times more as follows:
than the total number of static analysis results (TPs+FPs). 1) Installing WordPress and Plugins: First we need to
This it is very important when evaluating both the detection install and setup the WordPress application, and then install
capabilities (recall) of security tools and the ability to avoid the plugins. To avoid contamination from the presence of
false positives (precision). other plugins and the bias they may inflict to the database
during the execution of the tests, we install one clean instance
A. OBTAINING STATIC ANALYSIS DATA of the WordPress for each plugin.
The simultaneous use of diverse SA tools provides a means 2) Setting Up the Debugger: XDebug [41] is an extension
to increase the TPs over a single SA tool [37], although for PHP to assist with debugging and development. There-
this may come with an increase in the number of FPs. The fore, for gathering the execution trace files of the plugins we
dataset presented in [37] was built using five SA tools that activated the XDebug extension and configured it to create
were individually run to find SQLi and XSS vulnerabilities one debug file per HTTP request.
in 134 WordPress plugins. Then, their outputs were combined 3) Setting Up the Proxy: For web proxy we have chosen
and annotated, so each candidate vulnerability was manually OWASP ZAP [42], which is one of the world’s most popular
reviewed to identify if it was a TP or a FP. This dataset is free security tools. When it is used as a web proxy it is able
perfect to evaluate our blended methodology because with it to save the HTTP user interaction (requests and responses) in
we have a corpus with many vulnerabilities that have already its database for later analysis.
been manually annotated by experts. 4) Crawling the Applications: Since commercial auto-
To precisely define our scope, we focused our experimental mated crawlers are quite expensive, we have chosen to crawl
evaluation only on the first-order subset of SQLi vulnera- the plugins manually. This way, this task takes a fair amount
bilities of the dataset, which contains 462 TPs and 140 FPs of time, especially when dealing with plugins where the target
(subset of TNs), along with the detection results provided by features are only triggered automatically (e.g. when a timer
the SA tools, which are summarized in Table 1. The table expires) or externally (e.g when an user clicks on an email
includes the total number of potential SS in the plugins that link to continue an action initiated in the plugin to reset a
are about eight times more than the total number of SA results password) and not from the user interface of the plugin or

VOLUME 13, 2025 3145

 P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

TABLE 1. List of WordPress Plugins with first-order SQLi vulnerabilities. variables, identifying the executed SSs is done through a
sequential search of the data reported by the SA tools in the
Xtm file generated by XDebug.
Regarding the fourth step (crawling the applications),
it is important to mention that, a manual crawling process
requires an user with the ability to explore all elements
(e.g., links, buttons and input fields) in the plugins. The
user has to be familiar with WordPress and have a good
knowledge about the functionalities of the plugins. In fact,
different plugins add menu options in different parts of
the WordPress environment, therefore, the user has to be
aware of this. With the plugin installed and configured, the
user starts interacting with the login form by submitting
the admin credentials. WordPress starts the administration
dashboard. From this point, all visible buttons related with the
plugin (i.e., excluding general WordPress options) have to be
clicked, and all the forms with input elements must be filled
and submitted. Note, that for some plugins, we have to create
one or more WordPress/plugin users with different roles (e.g.,
Customer and Shop Manager for an e-commerce plugin) to
make it visible and allow accessing some parts of the plugin.
In these cases, the user crawling the plugin has to log in as
these users and navigating the plugin, executing every feature.
The time taken for the crawling step varies a lot, depending
on the complexity of the plugins. During our tests, we spend
between one minute for the simpler plugins, where we only
had to input one or two values, to about 15 minutes for the
most complex plugins where we had to navigate through
many pages and fill in several input values: choosing options
available from menus, buttons, and hyperlinks, data entry
on forms, checking emails to confirm operations, drag and
drop data items up and down. In other situations, due to
the dynamic interface of some of the plugins, we had to
previously insert specific data in the database in order to make
some user options available.

C. MAPPING HTTP REQUESTS WITH TRACE FILES

During this stage, we mapped the trace files with the
HTTP requests gathered by the proxy, using the timestamp
information they both have. The matching trace file is
software. The benefit of automated crawling is huge, reducing the first one created after the HTTP request, since we
the time, cost and resources required for the process, but web are executing all the actions sequentially, without threads.
crawling tools have low test coverage, as they are generic and Listing 1 shows several trace files names generated during
it is difficult to capture the business logic of the application the crawling process for the gallery-images.1.0.1
being crawled [40]. However, based on the requirements of a WordPress plugin. The trace files names have the fol-
web application, it is not difficult for an user to interact with lowing template name: ‘‘name_<timestamp(integer
the web application in order to run almost all features. part)>_<timestamp (decimal part)>__<HTTP
5) Identifying Vulnerable Executed SSs: XDebug trace request>’’. We removed from the HTTP request the prefix
files include lots of data about the execution of the web ‘‘localhost:81/’’ and replaced invalid characters for file
application. We used the XDebug Trace Manipulator (Xtm names (e.g., ‘‘/’’, ‘‘&’’, ‘‘?’’, ‘‘:’’) by an underscore ‘‘_’’
tool [43]) to parse, filter and format the trace file data and character. The decimal part of the timestamp has 6 digits of
create a new file with only the data that we need: the file granularity. We included the HTTP request in the trace file
name, LOC, variable names and values for the executed names for cross validation when mapping the trace file names
database functions (i.e., SQLi related SSs function calls, with HTTP requests.
like mysql_query). Since the results of the SA tools also The trace file name in line 4 of Listing 1 was mapped
include the vulnerable file name, LOC, SSs function and with the HTTP request in Listing 2 as this is the trace file

3146 VOLUME 13, 2025

P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

LISTING 3. URL of indirect call to execute part of a plugin code.

have to use the HTTP request obtained from the database

of the web proxy. The construction of the URL is more
complex in this case, since the URL is virtual and we need
to provide the correct WordPress environment variables that
will trigger the execution of the target vulnerable file with
the right parameters. It has two variations: 1) Consists of
providing a specific WordPress page, like the admin page
(e.g., wp-admin/[Link]?page=<name>. 2) Cor-
responds to an URL that is dynamically generated by the
LISTING 1. Partial list of trace files for the plugin WordPress interface. For example, hyperlinks to delete items
gallery-images.1.0.1. displayed in the web browser. Listing 3 shows an example
where the event identified by the deleteevent=2 of the
community-events-events page is deleted.

2) SQLMAP INTERNAL OPTIONS

Next, we list the SQLMap options that we used the most in
LISTING 2. Example of HTTP request log. the DA part.
Injection – Specifies which parameters to test for, the type
of back-end database, and custom injection payloads:
with the lowest timestamp ‘‘1555064689_307’’ (3 digits • dbms=VALUE. Selects the back-end DBMS. Since the
of granularity, milliseconds) that is greater than the timestamp WordPress uses MySQL, this was our choice.
of the HTTP request ‘‘1555064689_323058’’. In this • p. Testable parameter that is likely to be vulnerable.
case, the trace file was created just over 16 milliseconds • Injection payload prefix and suffix strings. These
(323058 − 307 × 1000) after the HTTP request. For cross options are very useful when the user knows the syntax
validation, we verify in the content of the trace file mapped if of the SQL statement [44], like in our work from the
the target SS was in fact executed. SA. Providing values for the prefix and suffix prevents
D. GENERATING THE DA CONFIGURATION the use of brute force trying all the combinations.
To test a vulnerable parameter, we can use automated Detection – Options to customize the detection phase:
• risk. The likelihood of a payload to damage the data
tools, such as SQLMap and bsqlbf-v2. Our choice was
SQLMap, because it is a free, fully-featured, configurable and integrity (1-3, default 1). Risk value 1 is innocuous for
widely used tool. The exact configuration must be tailored the majority of SQL injection points.
• level. Level of tests to perform (1-5, default 1 <
according to the specificity of the plugin and the vulnerability.
100 requests). This option limits the maximum number
1) TARGET VULNERABILITY EXECUTION of test cases tried to exploit the vulnerable parameter.
Depending on the WordPress plugin, we have two ways to Lines 7-8 in Listing 4 show and example of a DA config-
execute its code: 1) direct call and 2) indirect call, which is uration generated based on the vulnerability reported by the
the most common. A plugin may use any one of these ways or SA in the WordPress plugin quartz.1.01.1, located at
even both of them (each one for different parts of the code). the line 18 of the file all_quotes.php, which is listed
In our dataset there are 98 cases of direct call and 504 cases in line 3 of Listing 4. From the code of line 18 we extracted
of indirect call (Table 2). that the SS is $wpdb->query, the vulnerability is located
Direct call. The URL is composed by the main address and in the WHERE clause of a SQL DELETE statement and the
the GET parameters separated by the ‘‘?’’ symbol. The main vulnerable variable is $_REQUEST['quote'] enclosed in
address is composed by the base URL of the application (e.g., single quotes ('). From this we generate the following part of
[Link]) concatenated with the plugins folder the DA configuration ‘‘--prefix="'" --suffix="--
(wp-content/plugins) and the full path name of the abc’’ for creating an injection point (WHERE ID='2'
file of the plugin containing the vulnerability and param- <injection_point> -- abc') between the single
eters (e.g., ajaxgallery/utils/[Link]? quotes of the SQL statement.
itemId=3). Lines 4-6 of Listing 4 show data collected when the SS
The Indirect call is needed when the plugin has to be was executed during the crawling process. The item HTTP
called from within the WordPress environment. To obtain request contains the parameters and values. It includes the
the information necessary to execute this case, we also four parameters: page (quartz/all_quotes.php), action

VOLUME 13, 2025 3147

 P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

(delete), paged (1) and the quote (2), and the vul-
nerable parameter. The item SS contains the name of
the SS executed (wpdb->query) and the SQL state-
ments executed. From this we generate the ‘‘-u’’ and
the ‘‘-p’’ part of the DA configuration. From this we
generate the following part of the DA configuration:
‘‘-u “[Link]
wp-admin/[Link]?page=quartz/all_quotes
.php&action=delete&paged=1&quote=2’’ and ‘‘”
-p "quote"’’.
E. TESTING THE VULNERABILITY EXPLOITABILITY
To check if a vulnerability is exploitable, we executed
SQLMap with the configuration generated in the previous
stage (Section IV-D). For each vulnerability, SQLMap
LISTING 4. Example of PoC report for the quartz.1.01.1 plugin.
may provide two responses: 1) The tested parameter is
vulnerable. In this case, the output is saved in a text file.
TABLE 2. Overall results of the blended methodology.
2) All tested parameters do not appear to be injectable.
In this case, we restore the database, change one SQLMap
option at a time and re-execute SQLMap. When all test
options were tried without success, we consider that the
vulnerability is not exploitable (and we check it manually
to confirm). The details on how the options are mutated are
shown in the next paragraphs. The risk detection option
is the first one to be changed. We start by increasing it correctness of the results reported by the tools and by our
to risk=2 to test with time based techniques and then to approach. We also compare our approach with NAVEX [34].
risk=3 to test with OR-based techniques. The next option To understand the effectiveness of our approach, we discuss
to be changed is the level that has a maximum of 5. how the results compare with those obtained by directly using
Afterwards, we remove the prefix and suffix configurations, the outcome of the SA (like the URL, vulnerable parameters
which tests all prefix and suffix combinations. Finally, we try and parameters determined statically from the path) as the
to test with all injection techniques (technique=BEUSTQ). input of the DA tool.
The creation of the SQLMap configuration, including the
definition of the URL and the selection of the options is done A. OVERALL RESULTS
automatically by a set of Python scripts. Table 2 shows the overall results of our blended approach.
F. PoC REPORTING The first column indicates the type of call (direct or indirect)
After executing all previous stages for each vulnerability needed to execute the plugin code (see Section IV-D).
reported by the SA, a report fusing the data gathered from all For the TPs, column ‘‘SA’’ has the vulnerabilities correctly
the stages is generated. Listing 4 shows an example. The item detected in the SA stage, column ‘‘C’’ shows those from
‘‘SA’’ shows data from the SA, including the vulnerable file, which we could obtain the configuration parameters during
number of LOC, sensitive sink (wpdb->query), vulnerable the crawling, and column ‘‘DA’’ presents those that could be
parameter (quote). It also includes the LOCs (not all in exploited by the DA tool, which represent the vulnerabilities
this case) where we see that for triggering the SS is required that could be confirmed as such by our approach.
the parameter action with the value delete. The item For the True Negatives (TNs), column ‘‘SA’’ shows the
‘‘HTTP’’, shows the request triggering the sensitive sink with ones misclassified during the SA stage, column ‘‘C’’ shows
parameter’s values. The item ‘‘SS’’, contains the vulnerable those for which we could obtain the configuration parameters
code executed during the crawling of the application, with during the crawling, and column ‘‘DA’’ presents those that
the value 2 for the vulnerable parameter. The ‘‘SQLMap could not be exploited by the DA tool, so these are the
usage’’, shows the command used to exploit the vulnerable ones that our approach correctly indicates as not being a
code. Finally, the last item ‘‘SQLMap output’’ shows part of vulnerability.
the results of running the DA. The ‘‘Total’’ is the compound of the correct final
results: column ‘‘SA’’ are all the possible cases correctly
V. RESULTS AND DISCUSSION identified (either as a vulnerability or not) by the SA stage,
This section presents and discusses the results of the column ‘‘C’’ shows the situations for which we could obtain
empirical study presented in Section IV. As mentioned in the configuration parameters during the crawling, column
Section IV-A, for the purpose of this evaluation, we have ‘‘DA’’ presents those that were correctly identified by our
the vulnerability data annotated so we can identify the approach (either as a vulnerability or as false alarm), and

3148 VOLUME 13, 2025

P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

TABLE 3. Vulnerabilities exploited with direct calls. plugin that requires a PayPal account. Finally, the last one is
on multiple SQL statements in the sendit.2.1.0 plugin.
By analyzing its source code (used to insert new subscribers),
we found that the code block where the vulnerable code
is is only executed when the email of the new subscriber
does not exist in the database. In order to be able to exploit
the vulnerability, the DA would need to use an email that
does not exist in the database, which SQLMap cannot obtain
automatically.
Regarding the indirect call group (see Table 4), 95 (382-
287) SA results were wrongly excluded in the crawling
process, so they could not even be analyzed by SQLMap.
In fact, the crawling procedure was only able to cover
69.6% of the vulnerable code. As expected, it may be
difficult for the crawling to cover all the TPs, given the
specific needs of the plugins. For example, some plugins
require a commercial license to activate some features (e.g.,
42 known vulnerabilities of the js-appointment.1.5
plugin are located in its commercial part); the plug-
column ‘‘%’’ has the ratio of the results correctly identified by ins levelfourstorefront.8.1.14 and events-
our approach over the total number of positives given by the registration.5.44 require a PayPal account to make
SA stage. In short, by using our methodology, 76.7% (462) payments; some plugins, such as newsletter.3.6.4,
of the vulnerabilities reported by the SA could be confirmed execute cron jobs to periodically send emails; other plugins,
either as TPs or TNs, leaving only 140 (602-462) out of 602 like wp-championship.5.8, notify their users according
vulnerabilities to be checked manually (because our approach to the state of the database data. By having that commercial
could not execute them until the end, and not because of a license and the PayPal account configured it is likely that
wrong result) (see Table 2). Next, we discuss the reasons why these situations could be tested.
our solution failed to confirm these vulnerabilities. Considering the 287 TPs crawled by our indirect call
approach, SQLMap was unable to exploit 21 (287-266),
B. TESTING THE VULNERABILITY EXPLOITABILITY but there are several reasons for this. For instance,
Table 3 shows the results of the known vulnerabilities that SQLMap was not able to exploit nine vulnerabilities of the
were successfully exploited using direct calls and Table 4 events-registration.5.44 plugin because it has a
those using indirect calls. In these tables, we can see the CAPTCHA challenge. One missed vulnerability was found in
vulnerabilities found by the SA and the subset that SQLMap the newsletter.3.6.4 plugin. It occurs in an UPDATE
could confirm. The last column shows the total number statement which aims at resetting the values of one record of
of vulnerabilities that were confirmed to be exploitable. the table, but by injecting the payload " OR 1=1" in the id
The columns are also organized by type of vulnerable parameter, all records of the table are deleted. In general, it is
SQL statement: SELECT (S), INSERT (I), DELETE (D), difficult for DA tools to detect SQL commands that change
UPDATE (U) and M for multiple SQL statements. The the database, like INSERT, DELETE and UPDATE, because
multiple SQL statements identifies situations where different of the limited feedback they provide and the changes they
SQL statements share the same vulnerable variables in the introduce in the following tests. This could be improved by
same execution path. reseting the database prior of every test by SQLMap.
For the direct call group, Table 3 shows that our Three vulnerabilities unexploited in eventify.1.7.f
crawling process was able to collect all data required to plugin were due to the use of nonces to avoid the
successfully exploit 70 TPs. It also shows the 27 TPs replay of the same operation. Finally, when exploiting
(A column) that could be exploited based only on data one vulnerability in the ip-blacklist-cloud.3.4 the
collected statically. Therefore, parameters and concrete database data is deleted and the website crashes, preventing
values could be collected directly from the SA results or the detection.
collected by analyzing the code in the vulnerable execution
paths reported by the SA. For example, from the LOC C. CONFIRMING THE NON-EXPLOITABILITY OF FPS
if (isset($_POST['save']) in an execution path, In the direct call group, our approach was able to confirm all
we extract a required parameter POST save. We can also the FPs as such (Table 5), given that SQLMap was unable to
see that our approach failed to exploit 10 (80-70) confirmed exploit them. For the indirect call group (Table 6), it was also
vulnerabilities. One is in the ip-logger.3.0 plugin able to confirm all of them, except 14 situations that were
that requires an account in the [Link] web excluded earlier in the crawling process due to a CAPTCHA
site. Eight are in the levelfourstorefront.8.1.14 challenge.

VOLUME 13, 2025 3149

 P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

TABLE 4. Vulnerabilities exploited with indirect calls.

LISTING 5. Slice of code from the file [Link] of the

sermon-browser.0.43 plugin.

TABLE 5. FPs tested with direct calls.

TABLE 6. FPs tested with indirect calls.

Based on the annotated dataset (i.e., we know the FPs),

we expected SQLMap to fail in exploiting the FPs vulnera-
bilities reported by the SA. However, in a real world situation the inspection of the database before and after of the injection
we do not have annotations regarding FPs and when SQLMap to detect changes in the target records of the SQL statements.
is unable to exploit a vulnerability, we may question if it is a
FP reported by the SA or just a failure of SQLMap. Anyway, D. COMPARISON WITH ALTERNATIVE APPROACHES
we wanted to have some assurance that the non exploited By combining the results of Table 3 with Table 4, we have the
cases were FPs and, therefore, not exploitable or not trivial global outcome of the experiments applying our methodology
to exploit. This way, we analyzed the code where the FPs to obtain a confirmation of the truthfulness of the vulnerabil-
resulting from the SA were located and removed the security ities discovered by the SA tools. To understand the relevance
protections of the vulnerabilities. Then, we re-run SQLMap to of these results, we compared them with those from a scenario
exploit the modified code. In case of success, we considered using only SA and another scenario where we have the results
that the removed protections were, in fact, effective and that of the SA being fed directly (i.e., using data collected from
SA result was indeed a FP. This was how we confirmed the the vulnerable code reported by the SA) to DA (which is
results presented in Table 6. For example, 35 FPs in the plugin SQLMap) (Table 7):
sermon- browser.0.43 are due to a type cast (int) 1) SA: Having only the results of the SA, there are
of GET/POST parameters. Listing 5 provides an example of 602 (80+382+18+122) vulnerability alarms, but none con-
a type cast preventing SQLi vulnerabilities. After removing firmed, since no other mechanism was used for verification.
the type cast (int), in line 1333 SQLMap reports that the 2) SA+SQLMap: In this setup, the result of the SA is used
parameter mid was successfully exploited. This example to feed the SQLMap with the URL, vulnerable variables and
contains three SQL statements in different LOCs using the other parameters required for triggering the SS. Since this
variable $id. Confirming if all LOCs are vulnerable requires is the only information available, we can only consider the

3150 VOLUME 13, 2025

P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

TABLE 7. Comparison of the Effectiveness of our blended methodology. TABLE 8. Overall results for two NAVEX’s applications.

confirmation of 41 vulnerabilities of the direct calling group:

27 (total of column A in Table 3) + 14 (total of column SA
in Table 5).
3) Our Methodology: The SA tool was executed and the
outputs, along with the web application interaction data, were with the same web applications used in the NAVEX paper
fed to the SQLMap applying our methodology. This gives us and, this way, compare both results. However, from the
462 confirmations of the SA results (70+266+18+108, data 11 applications they used, one is from 2012 (osCommerce
from tables 3 to 6). 2.3.3), in three of them the code is no longer available
Table 7 shows a comparison of the effectiveness of the and the rest are older than 2010 (e.g., geccbblite (0.1)
three scenarios regarding their ability to confirm if the vulner- (Apr 11, 2005) and Schoolmate ( 1.5.4) (Nov 5, 2004). All
abilities are really exploitable. Column ‘‘SA stage’’ shows the the applications look very outdated and some applications
number of vulnerabilities detected by the SA tools. Column require the old PHP 4 and PHP 5 (current version is PHP
‘‘Crawling stage’’ shows the vulnerabilities that where 8). Because of the effort needed to find and annotate their
executed during the crawling process. Column ‘‘DA stage’’ vulnerabilities, we selected only two of the applications,
has the results of using the SQLMap after feeding it with SA an easy one and a complex one: the smallest in the dataset
data. The last column, ‘‘Manual review’’, depicts the number (geccbblite (0.1) and the largest in which NAVEX did not
and percentage of SA vulnerability results that must be exploit any vulnerability (SchoolMate 1.5.4). The first is
checked if we want to be sure that they are really exploitable. composed by 11 files and 323 LOCs and the second by
As expected, using only SA tools, all the results need to 63 files and 15375 LOCs.
be confirmed manually. This is the baseline. If we add the To apply our approach, we need to run one or more SA
DA tool to the equation (SA feeding directly DA, i.e. direct tools searching vulnerabilities in the applications. We started
calls), the need for manual analysis drops to 93.2%. However, with RIPS [38], configured for searching first order SQLi
if we consider our approach, this number drops to just 23.3%, vulnerabilities, for gathering SA outputs. We manually
which is a huge improvement. reviewed the results and concluded that all are TPs. During
Overall, these results highlight the relevance of the DA to this manual review, we found that the user input is used
improve the confidence on the output of the SA, and also that directly in the SQL statements in all vulnerabilities in the
deep knowledge about the user interaction is able to increase geccbblite application, which is easy to detect. The
this even further. Without using the necessary crawling data same happens in the SQLi vulnerabilities of the SchoolMate
to help configure the DA, a whole class of applications that application. These trivial vulnerabilities explain the lack of
have features that cannot be executed directly through a URL FPs reported by RIPS. Since RIPS alone reported more
would be left untested. In fact, the precise data required to vulnerabilities than NAVEX and the use of more SA
execute a specific potential vulnerability simplifies a lot the tools requires additional manual review efforts, we decided
process of DA to produce evidences that the vulnerability not to use additional tools. Continuing the application of
exists and can be exploited. our approach, we were able to generate exploits for all
vulnerabilities reported by the SA (RIPS tool).
E. COMPARING WITH NAVEX Table 8 shows the results obtained from the NAVEX
Though NAVEX [45] has been open-sourced, unfortunately, paper [45] and from the experiments with our methodology.
the source code (for taint analysis for vulnerability detection Overall, using our methodology, there are 160 vulnerabil-
and constraint collection) is incomplete and no longer main- ities reported by the SA. Where 23 (4+19) vulnerabilities
tained. Like other researchers (e.g., [46]), our attempt to reach could be exploited using direct calls and 129 that could
the authors failed. Meanwhile, a partially fixed version of the only be exploited with indirect calls. There are 8 SA results
tool can be found at [47], thanks to an anonymous author. where the crawling was not able to trigger the vulnerability.
This version is able to perform taint analysis, using a graph From these, NAVEX was able to generate exploits for
database, but in the joernsteps API, two fundamental only 4 vulnerabilities (column T of NAVEX), whereas our
and critical functions (start and findSinkLocation) approach was able to outperform it by far, generating exploits
that do the taint analysis are missing. Because of this, for 152 vulnerabilities (column T of our methodology).
it is impossible to directly reproduce the static analysis Like for NAVEX our approach depend of the SA results.
results present in the NAVEX’s paper and consequently the However, our approach can benefits of the results of diverse
dynamic analysis results. Given the impossibility of testing SA tools while NAVEX depends on its own SA results. For
the NAVEX tool by ourselves, we decided to use our approach instance, we compare the results of the SA of NAVEX with

VOLUME 13, 2025 3151

 P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

the SA results in the dataset we used for evaluation our that it shortens the number of the results that need to be
approach. checked manually to less than 1/4. Moreover, most of the
The implementation of NAVEX is based on many existing results that need to be checked manually come from lack
tools, most of which were extended to deal with PHP of external services configuration, that prevented the DA to
code: Joern, PHPJoern (code property graphs), Neo4j graph execute, not from the methodology itself. The whole process
database, Gremlin for graph traversals, Z3 solver, Z3-str2, can be performed in an automated fashion, optimizing both
crawler4j, Narcissus JavaScript engine, and XDebug. Addi- the speed of operation and its coverage, which is great for
tionally, some tools have dependencies with old resources security practitioners and penetration testers.
(e.g. Py2neo 2.0.7) that are difficult the obtain or that no Future work will focus on two main directions. One is
longer maintained. improving the automation of the crawling, by gathering data
from several sources (like the unit test data and the database)
F. THREATS TO VALIDITY
and improving the automated verification of the cases missed
There are some potential internal and external threats that
by the DA, which can be done using several DA tools as
may affect our study and the results.
each one produces different results, will lead to better results.
Internal threats: Dataset. Despite the results were
Another direction to improve the vulnerability detection is
obtained from a relevant number of WordPress plugins with
to consider also as potential vulnerabilities the SS executed
different sizes and functionalities, they only target SQLi
during the crawling process and not reported by the SA and
vulnerabilities and cannot necessarily be generalized to all
then proceed with the DA as in our proposed methodology.
web applications and classes of vulnerabilities. Crawling.
Our approach requires crawling the applications. Thus, the
REFERENCES
results are limited to how successful this task is. The use
[1] S. Rafique, M. Humayun, B. Hamid, A. Abbas, M. Akhtar, and
of automated tools minimize the burden of this process, K. Iqbal, ‘‘Web application security vulnerabilities detection approaches:
however they need to be carefully configured in order A systematic mapping study,’’ in Proc. IEEE/ACIS 16th Int. Conf. Softw.
to execute all the vulnerable code. Moreover, automated Eng., Artif. Intell., Netw. Parallel/Distrib. Comput. (SNPD), Jun. 2015,
pp. 1–6.
crawling has incomplete coverage especially for applications
[2] OWASP Found. (2021). OWASP Top 10—2021. Accessed: Oct. 5, 2021.
with complex and dynamic URLs. [Online]. Available: [Link]
External threats: DA tools. We used SQLMap to confirm [3] Application Security Statistics Report: The Case for DevSecOps, WhiteHat
the SA results. Using other DA tools does not necessarily Secur., 2017.
[4] Z. Guo, T. Tan, S. Liu, X. Liu, W. Lai, Y. Yang, Y. Li, L. Chen, W. Dong,
produce the same results because each tool has different and Y. Zhou, ‘‘Mitigating false positive static analysis warnings: Progress,
capabilities and settings. Unexploitability. When applying challenges, and opportunities,’’ IEEE Trans. Softw. Eng., vol. 49, no. 12,
our methodology, SQLMap was unable to exploit some pp. 5154–5188, Dec. 2023.
vulnerabilities. However, this does not mean that these [5] Accessed: Apr. 25, 2023. [Online]. Available: [Link]
com/business/resources/T141/reports/2023-data-breach-investigations-
situations are not exploitable at all. DA tools have many [Link]
tuning configuration options in order to be able to bypass [6] Accessed: Mar. 18, 2022. [Online]. Available: [Link]
several security measures that may be used by the application blog/best-practices-to-secure-web-applications-from-vulnerabilities
[7] M. D. Ernst, ‘‘Static and dynamic analysis: Synergy and duality,’’ in Proc.
under test. Although all these options may be very powerful, ICSE Workshop Dyn. Anal. (WODA), Jan. 2003, pp. 24–27.
sometimes a vulnerability is only exploitable in a concrete [8] G. Deepa and P. S. Thilagam, ‘‘Securing Web applications from injection
state of the application, like having specific data in the and logic vulnerabilities: Approaches and challenges,’’ Inf. Softw. Technol.,
database or having a PayPal account. These cases require vol. 74, pp. 160–180, Jun. 2016.
[9] D. Balzarotti, M. Cova, V. Felmetsger, N. Jovanovic, E. Kirda, C. Kruegel,
manual testing based on deep knowledge on the inner and G. Vigna, ‘‘Saner: Composing static and dynamic analysis to validate
workings of the application. sanitization in web applications,’’ in Proc. IEEE Symp. Secur. Privacy (SP),
May 2008, pp. 387–401.
VI. CONCLUSION [10] M. Althunayyan, N. Saxena, S. Li, and P. Gope, ‘‘Evaluation of black-box
In this paper, we presented a blended approach using SA, DA Web application security scanners in detecting injection vulnerabilities,’’
Electronics, vol. 11, no. 13, p. 2049, Jun. 2022.
and application interaction for vulnerability detection. After
[11] W. G. J. Halfond and A. Orso, ‘‘AMNESIA: Analysis and monitoring
running the SA, the target application is executed while its for NEutralizing SQL-injection attacks,’’ in Proc. 20th IEEE/ACM
interaction is being recorded so it can provide the DA with the Int. Conf. Automated Softw. Eng., New York, NY, USA, Nov. 2005,
necessary intelligence to increase its detection capabilities. pp. 174–183.
[12] Z. Su and G. Wassermann, ‘‘The essence of command injection attacks
The outcome of this methodology is a framework that is in web applications,’’ in Proc. Conf. Rec. 33rd ACM SIGPLAN-
capable of discovering a large set of proven to be exploited SIGACT Symp. Princ. Program. Lang., New York, NY, USA, Jan. 2006,
vulnerabilities in web applications. pp. 372–382.
[13] A. Ciampa, C. A. Visaggio, and M. Di Penta, ‘‘A heuristic-based approach
Experimental evaluation considering a set composed by a
for detecting SQL-injection vulnerabilities in Web applications,’’ in Proc.
population of 602 SA results (462 SQLi TPs and 140 FPs) ICSE Workshop Softw. Eng. Secure Syst., New York, NY, USA, May 2010,
from 49 WordPress plugins, show that the proposed approach pp. 43–49.
was able to outperform by far current procedures using only [14] I. Lee, S. Jeong, S. Yeo, and J. Moon, ‘‘A novel method for SQL
injection attack detection based on removing SQL query attribute
SA to search for vulnerabilities. In fact, it was able to confirm values,’’ Math. Comput. Model., vol. 55, nos. 1–2, pp. 58–68,
most of the SA results as being either TPs or FPs, in a way Jan. 2012.

3152 VOLUME 13, 2025

P. Nunes et al.: Blending Static and Dynamic Analysis for Web Application Vulnerability Detection

[15] D. Mitropoulos, P. Louridas, M. Polychronakis, and A. D. Keromytis, [38] J. Dahse and T. Holz, ‘‘Simulation of built-in PHP features for precise static
‘‘Defending against Web application attacks: Approaches, challenges and code analysis,’’ in Proc. Netw. Distrib. Syst. Secur. Symp. (NDSS), 2014,
implications,’’ IEEE Trans. Dependable Secure Comput., vol. 16, no. 2, pp. 1–15.
pp. 188–203, Mar. 2019. [39] B. Johnson, Y. Song, E. Murphy-Hill, and R. Bowdidge, ‘‘Why don’t
[16] D. Appelt, C. D. Nguyen, L. C. Briand, and N. Alshahwan, ‘‘Automated software developers use static analysis tools to find bugs?’’ in Proc. 35th
testing for SQL injection vulnerabilities: An input mutation approach,’’ Int. Conf. Softw. Eng., May 2013, pp. 672–681.
in Proc. Int. Symp. Softw. Test. Anal., New York, NY, USA, Jul. 2014, [40] J. Shahid, M. K. Hameed, I. T. Javed, K. N. Qureshi, M. Ali, and N. Crespi,
pp. 259–269. ‘‘A comparative study of Web application security parameters: Current
[17] L. Lei, X. Jing, L. Minglei, and Y. Jufeng, ‘‘A dynamic SQL injection trends and future directions,’’ Appl. Sci., vol. 12, no. 8, p. 4077, Apr. 2022.
vulnerability test case generation model based on the multiple phases [41] XDebug Extension for PHP. Accessed: Jan. 30, 2018. [Online]. Available:
detection approach,’’ in Proc. IEEE 37th Annu. Comput. Softw. Appl. Conf., [Link]
Jul. 2013, pp. 256–261. [42] Accessed: Feb. 4, 2018. [Online]. Available: [Link]
php/OWASP_Zed_Attack_Proxy_Project
[18] S. Rawat, D. Ceara, L. Mounier, and M.-L. Potet, ‘‘Combining static and
[43] XTM, XDebug Trace Manipulator. Accessed: Jan. 28, 2018. [Online].
dynamic analysis for vulnerability detection,’’ 2013, arXiv:1305.3883.
Available: [Link]
[19] Accessed: Oct. 31, 2024. [Online]. Available: [Link] [44] SQLMap. Accessed: Nov. 5, 2018. [Online]. Available: [Link]
technologies/overview/programming_language [45] Accessed: Mar. 1, 2021. [Online]. Available: [Link]
[20] Accessed: Oct. 31, 2024. [Online]. Available: [Link] navex
technologies/overview/content_management/all [46] P. Li, W. Meng, K. Lu, and C. Luo, ‘‘On the feasibility of automated built-
[21] G. Díaz and J. R. Bermejo, ‘‘Static analysis of source code security: in function modeling for PHP symbolic execution,’’ in Proc. Web Conf.,
Assessment of tools against SAMATE tests,’’ Inf. Softw. Technol., vol. 55, New York, NY, USA, Apr. 2021, pp. 58–69.
no. 8, pp. 1462–1476, Aug. 2013. [47] (2020). Accessed: Feb. 1, 2022. [Online]. Available: [Link]
[22] P. J. C. Nunes, ‘‘Blended security analysis for web applications: UUUUnotfound/Navex_fixed
Techniques and tools,’’ Ph.D. thesis, Universidade de Coimbra,
PAULO NUNES received the Ph.D. degree in
Coimbra, Portugal, 2022. [Online]. Available: [Link]
information science and technology from the
handle/10316/100340?mode=simple
University of Coimbra, in 2022. He has been an
[23] A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr, ‘‘Basic concepts
Adjunct Professor with the Department of Com-
and taxonomy of dependable and secure computing,’’ IEEE Trans.
puter Science, Polytechnic Institute of Guarda
Dependable Secure Comput., vol. 1, no. 1, pp. 11–33, Jan. 2004.
(IPG), since 1998. He has been a Researcher with
[24] I. V. Krsul, ‘‘Software vulnerability analysis,’’ Ph.D. thesis, Purdue Univ.,
the Research Unit for Inland Development (UDI),
West Lafayette, IN, USA, 1998.
IPG, Since 2007. From September 2014 to May
[25] Plugin Handbook. [Online]. Available: [Link]
2022, he was a Research Student with the Centre
plugins/
for Informatics and Systems of the University of
[26] M. K. Gupta, M. C. Govil, and G. Singh, ‘‘Static analysis approaches
Coimbra (CISUC). Since 2022, he has been with the CISUC as a Former
to detect SQL injection and cross site scripting vulnerabilities in web
Member. His interests include web security assessment and benchmarking,
applications: A survey,’’ in Proc. Int. Conf. Recent Adv. Innov. Eng.
(ICRAIE), May 2014, pp. 1–5. subjects in which he has authored or co-authored several papers in refereed
conferences and journals.
[27] A. S. Choudhary and M. L. Dhore, ‘‘CIDT: Detection of malicious code
injection attacks on web application,’’ Int. J. Comput. Appl., vol. 52, no. 2, JOSÉ FONSECA received the Ph.D. degree in
pp. 19–26, Aug. 2012. informatics engineering from the University of
[28] Accessed: Oct. 5, 2019. [Online]. Available: [Link] Coimbra, in 2011. Since 2005, he has been with the
com/knowledge/knowledgebase/SQLi CISUC as a Researcher. He has been a Professor
[29] Y. Wang, D. Wang, W. Zhao, and Y. Liu, ‘‘Detecting SQL vulnerability of computer science related courses with the
attack based on the dynamic and static analysis technology,’’ in Proc. IEEE Polytechnic Institute of Guarda, since 1993. He is
39th Annu. Comput. Softw. Appl. Conf., vol. 3, Jul. 2015, pp. 604–607. the author or co-author of more than 30 papers
[30] K. Umar, A. Bakar Md Sultan, H. Zulzalil, N. Admodisastro, and in refereed conferences and journals. His research
M. T. Abdullah, ‘‘SQL injection attack roadmap and fusion,’’ Indian J. Sci. on vulnerability and attack injection was granted
Technol., vol. 9, no. 28, pp. 1–8, Jul. 2016. with the DSN’s William Carter Award of 2009,
[31] J. Fonseca, M. Vieira, and H. Madeira, ‘‘Testing and comparing web sponsored by the IEEE Technical Committee on Fault-Tolerant Computing
vulnerability scanning tools for SQL injection and XSS attacks,’’ in Proc. and the IFIP Working Group on Dependable Computing and Fault Tolerance
13th Pacific Rim Int. Symp. Dependable Comput. (PRDC), Dec. 2007, (WG 10.4).
pp. 365–372.
[32] L. Kree, R. Helmke, and E. Winter, ‘‘Using semgrep OSS to find OWASP MARCO VIEIRA (Member, IEEE) is currently a
top 10 weaknesses in PHP applications: A case study,’’ in Proc. 21st Int. Professor of computer science with the University
Conf. Detection Intrusions Malware, Vulnerability Assessment (DIMVA), of North Carolina at Charlotte. Before joining
in Lecture Notes in Computer Science, vol. 14828, Lausanne, Switzerland. UNC Charlotte, in 2023, he was a Professor with
Cham, Switzerland: Springer, Jul. 2024, pp. 64–83. the University of Coimbra. His research interests
[33] T. Ball, ‘‘The concept of dynamic analysis,’’ ACM SIGSOFT Softw. Eng. include dependable computing, dependability and
Notes, vol. 24, no. 6, pp. 216–234, Nov. 1999. security assessment and benchmarking, software
[34] A. Alhuzali, R. Gjomemo, B. Eshete, and V. N. Venkatakrishnan, security, fault and vulnerability injection, failure
‘‘NAVEX: Precise and scalable exploit generation for dynamic web prediction, static analysis, and software testing,
applications,’’ in Proc. 27th USENIX Secur. Symp. (USENIX Secur.), subjects in which he authored or co-authored more
Baltimore, MD, USA, Aug. 2018, pp. 377–392. than 250 works in refereed conferences and journals. He is the Chair of the
[35] RIPS Technologies. Accessed: Apr. 17, 2019. [Online]. Available: https:// IFIP WG 10.4 on Dependable Computing and Fault Tolerance, an Associate
[Link]/features Editor of IEEE TRANSACTIONS ON DEPENDABLE AND SECURE COMPUTING, the
[36] I. Medeiros, N. F. Neves, and M. Correia, ‘‘Automatic detection and Steering Committee Vice-Chair of the IEEE/IFIP International Conference
correction of web application vulnerabilities using data mining to predict on Dependable Systems and Networks, and member of the Steering
false positives,’’ in Proc. 23rd Int. Conf. World Wide Web, Apr. 2014, Committee of the IEEE International Symposium on Software Reliability
pp. 63–74. and Engineering and of the Latin-American Symposium on Dependable
[37] P. Nunes, I. Medeiros, J. C. Fonseca, N. Neves, M. Correia, and M. Vieira, and Secure Computing. He served as the Program Chair for the major
‘‘Benchmarking static analysis tools for web security,’’ IEEE Trans. Rel., conferences in the dependable computing area.
vol. 67, no. 3, pp. 1159–1175, Sep. 2018.

VOLUME 13, 2025 3153