2025 55th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)

Detecting Code Vulnerabilities using LLMs

Larry Huynh∗ , Yinghao Zhang∗ , Djimon Jayasundera∗ , Woojin Jeon† ,
2025 55th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) | 979-8-3315-1201-9/25/$31.00 ©2025 IEEE | DOI: 10.1109/DSN64029.2025.00047

Hyoungshick Kim† , Tingting Bi∗ and Jin B. Hong∗‡

∗ The
University of Western Australia, Perth, Australia
Emails: {[Link], [Link], [Link]}@[Link]; {23072214, 23061402}@[Link]
† Sungkyunkwan University, Seoul, South Korea

Emails: {dnwls0116, hyoung}@[Link]

‡ Corresponding author

Abstract—Large language models (LLMs) have emerged as code patterns and structures. However, their application to
a promising tool for detecting code vulnerabilities, potentially vulnerability detection remains in early stages, with mixed
offering advantages over traditional rule-based methods. This pa- results. Some studies show LLMs outperforming static code
per proposes an enhanced framework for vulnerability detection
using LLMs, incorporating various prompt engineering strate- analyzers [10], while others highlight limitations in security
gies to improve performance. We evaluate several techniques, concept understanding and high false-positive rates [11], [12],
including role-based prompting, zero-shot chain-of-thought, and underscoring the need for further research.
structured prompting approaches, on the DiverseVul dataset of Prompt engineering [13] has emerged as a strategy to
C/C++ vulnerabilities. Our experiments assess the framework’s
performance across different code structures, contextual informa- enhance LLM performance across various tasks, including vul-
tion levels, and LLM capabilities. Our results show that using nerability detection [14]. However, while structured prompts
our dynamic prompt engineering technique, you can improve can improve response quality, they do not inherently pro-
the F1 score by up to 100% with GPT-3.5, a widely used LLM vide models with deeper understanding of code semantics or
model. We also observe that GPT-4o, Gemini 2.0 Flash, and security-specific contexts. As a result, prompt-based methods
Meta Llama 3.1 generally outperform GPT-3.5, and all models
are very poor when it comes to correctly identifying the type alone may struggle with vulnerabilities that require nuanced
of vulnerability in the code, with the best F1 score of 0.16 reasoning beyond pattern recognition. To address this lim-
observed. However, our follow-up experiments on LLM-based itation, integrating prompt engineering with context-aware
vulnerability correction (i.e., patching) show a 45.77% success analysis is essential for improving detection performance,
rate using GPT-4o, demonstrating promising results in leveraging particularly when vulnerabilities span multiple functions or
LLMs for enhancing software security and providing insights into
optimizing prompt engineering for vulnerability detection tasks. rely on implicit program behaviors.
Existing studies on LLM-based vulnerability detection often
Index Terms—Code Vulnerability, CWE, Large Language rely on datasets with isolated vulnerable snippets, which
Models, Prompt Engineering simplifies the detection process. However, in practice, vul-
nerabilities are embedded within larger code functions or
I. I NTRODUCTION
contexts with unknown locations, creating a gap between
Software vulnerabilities remain a critical security concern, current datasets and real-world scenarios. Addressing this gap
with early detection during development being crucial for requires methods capable of handling vulnerabilities across
system security, reliability, and robustness. Common vulner- varying code contexts, including entire functions or multi-
abilities like buffer overflows in C/C++ can lead to severe function structures.
security breaches if not promptly addressed [1]. Traditional In this paper, we propose an enhanced framework for code
detection methods, including manual code review and rule- vulnerability detection (CVD) using LLMs with prompt engi-
based static analysis tools, often prove time-consuming and neering strategies. Our approach addresses current LLM lim-
error-prone [2], [3], highlighting the need for more sophisti- itations through carefully crafted prompts and context-aware
cated automated solutions. Recent advancements in artificial analysis. To investigate how LLMs detect code vulnerabilities,
intelligence, particularly deep learning [4] and other AI tech- we focus on two critical dimensions: (1) prompt engineering—
niques [5], offer promising avenues for enhancing vulnerability what we say when instructing the LLM, and (2) context—what
detection through efficient analysis of vast code repositories and how much information we provide about the code. We
and adaptation to emerging threats. compare various prompting strategies (e.g., chain-of-thought,
Large Language Models (LLMs) have emerged as powerful role-based prompts) and contextual information levels (e.g.,
tools in software engineering, including code generation, bug snippet-only vs. full-file) on the DiverseVul dataset [15] of
fixing, and documentation writing [6]–[8]. Their ability to C/C++ vulnerabilities.
understand and generate human-like text and code opens new
We address the following research questions:
possibilities for automated vulnerability detection. LLMs can
generalize from training data to perform well on tasks they • RQ1: Do different prompt engineering strategies affect
weren’t explicitly trained for [9] and can identify complex the performance of vulnerability detection using LLMs?

2158-3927/25/$31.00 ©2025 IEEE 401

DOI 10.1109/DSN64029.2025.00047
Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 • RQ2: Do the capabilities of different LLM versions
influence vulnerability detection accuracy?
• RQ3: What is the impact of varying levels of code context
on detection performance?
• RQ4: Can LLMs effectively detect vulnerabilities found
in single-function and multi-function code structures?
Our contributions include:
• A novel framework for LLM-based vulnerability detec- Fig. 1: Overview of the CVD Framework.
tion, incorporating six prompt engineering techniques and
their comprehensive evaluation
• Analysis of code context and LLM capabilities on de- vulnerability detection in Section II-D. The evaluation criteria
tection performance, including comparisons between top and metrics used are shown in Section II-E.
25 Common Weaknesses Enumeration (CWE) vulnera-
bilities and others A. Pre-study: A User Study
• Insights into LLMs’ effectiveness in handling vulnera- We conducted an online survey via Prolific to investi-
bilities embedded within full functions and varying code gate attitudes towards LLMs in secure coding. The survey
contexts, addressing the gap between current datasets and targeted respondents from English-speaking countries (US,
real-world scenarios UK, Canada, and Australia) to ensure linguistic consistency
• Identification of challenges and opportunities in using with the primarily English-trained LLM models [16], thus
LLMs for both vulnerability detection and correction mitigating potential language-related biases.
Our work advances automated vulnerability detection and To ensure data quality and relevance, we implemented a
provides insights for researchers, practitioners, and the general two-step validation process: screening participants for LLM
public using LLMs for coding projects. programming experience and requiring a validation code upon
The remainder of this paper is organized as follows. Section survey completion. This rigorous approach yielded 105 valid
II describes our methodology, Section III presents results, participants, providing 1,470 unique responses across 14 ques-
Section IV discusses implications, Section V addresses validity tions. The survey had a median completion time of 5 minutes,
threats, Section VI reviews related works, and Section VII with participants compensated at a flat rate of $2.50 USD.
concludes with future directions. Our study strictly focused on the use of LLM in programming
contexts, adhering to ethical research practices by collecting
II. M ETHODOLOGY only necessary information and protecting participant privacy.
The questions themselves span three main categories: (1)
We present an approach to CVD that leverages LLMs
participant details, (2) the general use of LLMs for cod-
and various prompt-based engineering strategies. The core of
ing, and (3) the use of LLMs for secure coding practices.
our framework involves providing the LLM with carefully
Questions assessed participants’ frequency of LLM use for
crafted prompts containing potentially vulnerable code, ac-
code assistance (from 0% to 100% of coding time), as well
companied by contextual information such as comments and
as their confidence in and satisfaction with LLM-generated
docstrings. By combining prompt engineering with context-
code, both generally and specifically regarding secure code
aware analysis, our approach ensures that LLMs not only
writing, identification, and correction. We also explored par-
leverage structured instruction but also interpret vulnerabilities
ticipants’ prompting strategies for secure code writing and
within the broader scope of real-world code. This enables a
review. Respondents could select multiple approaches from a
more nuanced assessment beyond pattern matching, improving
list, including zero-shot, few-shot, chain-of-thought, recursive
the model’s ability to detect complex vulnerabilities spanning
criticism and improvement, security expert persona, explicit
multiple functions or implicit logic. The framework capturing
security requirements, and vulnerability-aware prompting. The
the workflow is shown in Figure 1. This adaptive approach
complete set of questions and anonymized responses are
allows for continuous refinement of the detection process,
available on our GitHub.
ensuring that the system’s accuracy and robustness improve
over time. The code is also provided on GitHub1 . The key survey questions (SQ) asked are summarized below.
Before populating contents for each code vulnerability de- • SQ1: Which LLM do you prefer for coding assistance?

tection phase, we conducted a pre-study survey to understand • SQ2: How satisfied are you with code written by LLMs

user perceptions and preferred approaches currently taken (as for your programming tasks?
shown in Section II-A). Then, we present the dataset used in • SQ3: How often do you consider the security of the code

Section II-B. The details of prompt engineering techniques are you write?
found in Section II-C. Using the dataset, prompt engineering • SQ4: How much do you trust that the code written by

techniques, and the selected LLMs, we describe the code the LLM is secure?
The survey results guide our selection of LLM models and
1 [Link]
prompt strategies for evaluation, ensuring our research aligns

402

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 TABLE I: LLM Secure Coding Pre-Survey; Key Results. TABLE II: Summary Statistics of Datasets Considered.
Question Never Occasionally Frequently Always GitHub
Dataset Projects CWEs Functions Labels
SQ2 2.9% 46.6% 48.6% 1.9% Link
SQ3 8.6% 41.9% 25.7% 23.8% Devign [19] 2 N/A 26037 No Yes
SQ4 18.1% 48.6% 28.6% 4.7% BigVul [20] 348 91 264919 Yes Yes
CrossVul [21] 498 107 134126 Yes No
CVEFixes [22] 564 127 168089 Yes Yes
DiverseVul [15] 797 150 330492 Yes Yes
with real-world behaviors in secure coding. For SQ1, our
survey shows that 58.1% of participants preferred using Ope-
nAI’s GPT series, with GPT-4 models being the most popular
ception and actual LLM performance in secure coding tasks
with 48.6%. The next most popular vendor was Microsoft
highlights a potential risk: users may be overestimating the
with the next highest proportions being 10.5% for Copilot,
security capabilities of these models, potentially leading to
followed by 9.5% for OpenAI GPT-3.5. GPT-4o has also been
the introduction of vulnerabilities in their code.
noted to be more performant than GPT-4 over a variety of
In terms of techniques used with LLMs by users, we observe
benchmark tasks [17]. The next preferred models were Claude
that the chain-of-thought prompting technique was most pop-
3.5 Sonnet (8.7%) and Google Gemini (7.7%). We include
ular for both writing secure code and reviewing code security,
Gemini Flash 2.0 as Gemini models are comparatively popular
preferred by 52.4% and 43.8% of participants, respectively.
to Claude 3.5 Sonnet, but is available at approximately 30x
The other prompting strategies were almost equally preferred.
reduced API cost ($3 vs $0.1 per million input tokens; Claude
Thus, we also investigate the effect of different prompting
vs. Gemini), making it an attractive alternative to currently
techniques and their effects on secure coding, which are further
available services2 . The next provider afterwards is Meta with
detailed in Section II-C.
Llama 3(1.9%). Despite its lower usage, we include this model
as it offers a free, locally-deployable option valuable for secure B. Dataset
coding applications requiring data privacy. We thus conduct
To validate our method, we utilized the DiverseVul dataset
experiments using GPT-4o, GPT-3.5 (as a low-cost alternative
[15], a comprehensive aggregation of existing C/C++ vulner-
to GPT-4o), Google Gemini Flash 2.0, and Meta’s Llama 3.
ability datasets sourced from various GitHub projects. Within
Unfortunately, there is a lack of accessible API for Microsoft
this dataset, each datapoint corresponds to a specific GitHub
Copilot, so it was omitted in our experiments.
commit designed to address one or more identified code
Table I summarizes the response for each survey question.
vulnerabilities, each classified by their CWE code [18].
Various observations are made that are critical to users when
We use the DiverseVul dataset rather than other similar
it comes to LLMs and secure coding. First, users are generally
options, shown in Table II, for the following reasons:
over-reliant on LLMs, where only 3% of participants indicated
that they stopped using an LLM for coding after experiencing • Comprehensiveness: It aggregates vulnerabilities in

it at least once. This means over 97% are using LLMs to 330,492 functions from 797 GitHub projects, encompass-
assist with code writing. Secondly, users generally trust the ing a wider range of code patterns and vulnerability types
code outputs provided by LLMs in both general and secure compared to individual datasets like ReVeal or BigVul.
coding contexts. An overwhelming 91.4% of participants are • Representation: The dataset captures a diverse range of

concerned with the security of the code they write (i.e., SQ3), 150 CWEs, with many of the most common CWE labels
but 81.9% trust the security of LLM-generated code to some being present. Further, there is a strong overlap with the
degree (i.e., SQ4). This implies that users have a high level CWEs contained in MITRE’s Top 253 , including CWE-
of confidence in LLMs for secure coding tasks. Furthermore, 787 (no. 1), CWE-416 (no. 4), CWE-20 (no. 6), and
the survey reveals a notable difference in how users perceive CWE-125 (no. 7).
the capabilities of GPT-3.5 and GPT-4o in secure coding • Real-World Relevance: The strong representation of

assistance. Among GPT-3.5 users, only 10% reported trusting common and critical CWEs enhances the dataset’s rel-
its capabilities “Frequently” or “Always.” In contrast, 38.1% evance for real-world vulnerability detection tasks. Fur-
of GPT-4o users expressed this level of trust. This may be ther, unlike synthetic datasets such as SARD, DiverseVul
because participants generally view GPT-4o as more capable sources vulnerabilities from actual GitHub commits, en-
than GPT-3.5, suggesting a higher level of trust in more suring their relevance to practical development.
advanced models for security-related tasks. • Recency: Published in 2023, DiverseVul is a relatively

However, this reliance and trust in LLMs for secure coding recent dataset compared to others like Devign (2019) or
contrasts with findings from previous studies, which have CVEFixes (2021). This recency captures the latest vul-
shown that the performance of LLMs in secure code assis- nerability patterns and trends, making it more applicable
tance is highly varied with some results indicating significant to current software development practices.
shortcomings [11], [12]. This discrepancy between user per- • Full Context Availability: Importantly, DiverseVul al-
lows us to retrieve the full code context for each vul-
2 API pricing for Claude: [Link]
and Gemini models: [Link] 3 [Link]

403

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 nerability. In practice, developers analyze code within its vulnerability detection. Each data point (or code input) is
broader context, not in isolation. Most existing datasets integrated with the relevant prompt strategy, producing six
do not offer the ability to reconstruct full code context (one baseline and five prompt engineering) versions of the
from raw data. DiverseVul, however, retains the necessary DiverseVul dataset.
commit-level information, allowing us to link each vul-
nerable function to its surrounding code. This is essential C. Prompt Engineering Strategies
for evaluating LLMs under conditions that reflect real- We explore several prompt engineering techniques to en-
world usage, as vulnerabilities often span multiple code hance the performance of LLMs in detecting code that leads
locations, and evaluating only on isolated snippets would to software vulnerabilities (which we will denote as code
not capture the complete vulnerability landscape. vulnerabilities). These techniques are designed to improve
Other datasets, such as Devign, BigVul, and CrossVul, the model’s reasoning capabilities and the quality of its
primarily contain pre-extracted code snippets targeting specific outputs. The following prompt engineering strategies were
vulnerable functions without broader context. While valuable implemented and evaluated:
for analyzing single-function vulnerabilities, they lack the • Baseline: The LLM is given the affected code as its only

full context needed to evaluate LLM performance in realistic input - no prompt engineering is applied here. This serves
scenarios. If a developer has already pinpointed the exact as a baseline for the other prompt engineering strategies.
vulnerable function, they likely have enough information to Example Baseline prompt:
address it without an LLM’s help. This makes DiverseVul the Analyze the provided code and determine if the code
only suitable dataset for our analysis, as other datasets do is vulnerable or safe, if vulnerable identify the most
not provide the code context required to emulate real-world obvious CWE.
vulnerability detection. [CODE]
To enhance the utility of the dataset, we apply several • Role-based Prompting: The LLM is instructed to as-
augmentation steps: sume the role of a cybersecurity expert. This approach has
• Full Context Retrieval: Leveraging GitHub’s API, we demonstrated superior performance compared to Chain-
systematically link each vulnerability-fixing commit to its of-Thought (CoT) and zero-shot CoT in certain reasoning
associated source code files. This ensures that the models tasks, while still eliciting CoT-like behavior in the LLM’s
have access to the complete context surrounding each responses [23]. Example Role-based prompt:
vulnerability. From now on, you are an excellent cybersecurity
• Data Normalization: To optimize data organization and expert who can analyze given code and determine
eliminate redundancy, we transform the original flat struc- if it is vulnerable or safe, and if vulnerable, identify
ture of the dataset into a hierarchical format. the related CWE.
[Baseline prompt]
• Zero-shot Chain-of-Thought (CoT): This strategy en-
hances the prompt with the phrase “Let’s think step by
step,” encouraging the LLM to outline its reasoning in
stages [24]. This approach guides the model to engage in
sequential reasoning, which has been shown to improve
accuracy in tasks requiring logical deduction without
needing few-shot exemplars. Example Zero-shot CoT
prompt:
[Baseline prompt]
Let’s think step by step.
• APE Zero-shot CoT: APE (Automatic Prompt Engi-
Fig. 2: Structure of DiverseVul Dataset. neering) zero-shot CoT builds upon the zero-shot CoT
method with a refined prompt that encourages a more
We further refine the dataset through a series of post- structured reasoning process. The APE framework auto-
processing operations. We separate data points into vulnerabil- mates prompt selection by iterating through options and
ities that existed across multiple functions and those that were assessing outcomes across tasks; in their original study,
contained within a single function. We also remove data points the authors identified “Let’s work this out in a step-by-
lacking a CWE identifier or containing multiple CWEs. We step way to be sure we have the right answer” as the
also remove commits that changed more than one file to ensure optimal prefix for complex reasoning tasks using their
all functions examined are relevant to the CWE identifier. APE-CoT strategy [25]. This prompt was then shown
We then apply our prompt engineering strategies (Section to outperform both standard zero-shot CoT and human-
II-C) to the dataset to investigate the capabilities of each crafted CoT prompts across various reasoning tasks. In
strategy in enhancing the performance of LLM-based code our study, APE zero-shot CoT was selected to assess

404

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 whether automated prompt refinements could enhance information in a structured format can help the LLM
the LLM’s consistency in detecting code vulnerabilities. improve its vulnerability analysis.
Example APE-CoT prompt: Example Prediction prompt:
[Baseline prompt] Analyze the provided code below and return two
Let’s work this out in a step-by-step way to be sure lines of text. The first line states the programming
we have the right answer. language of the code. The second line is a list of
• CO-STAR: The prompt follows a defined structure that potential vulnerabilities based on the code’s func-
uses XML tags to define Context, Objective, Style, Tone, tionality ONLY. Do not return any code.
Audience, and Response [26]. This technique has proven Code to analyze: [CODE]
effective in improving LLMs’ content generation capa- Example Dynamic prompt:
bilities across various tasks. Example CO-STAR prompt: <CONTEXT>
<CONTEXT> You are asked to accurately detect if a code is
You are asked to accurately detect if a code is vulnerable or safe. The vulnerabilities are related
vulnerable or safe. The vulnerabilities are related to the CWE list.
to the Common Weakness Enumeration (CWE) list. </CONTEXT>
</CONTEXT> <OBJECTIVE>
<OBJECTIVE> [Baseline prompt]
[Baseline prompt] </OBJECTIVE>
</OBJECTIVE> <HINT>
<STYLE> Ensure your analysis is accurate. Use the following
Provide answers based on the Common Weakness information to assist your analysis:
Enumeration descriptions and IDs. [Prediction response]
</STYLE> </HINT>
<TONE> These prompting techniques are selected due to their zero-
Logical. shot property. Unlike other prompting techniques, which re-
</TONE> quire a few-shot approach, these techniques can evaluate
<AUDIENCE> LLM’s performance and the influence of prompting with-
Your response will be read by cybersecurity experts out extra information besides the given code. Additionally,
to assess the accuracy of your response. few-shot approaches require a set of vulnerability detection
</AUDIENCE> examples. Given that our dataset only contains code and
<RESPONSE> CWE pairs, such data would not serve as good examples
Analysis of the code and your determination of due to the lack of vulnerability analysis. Due to this limi-
whether the code is vulnerable or safe. If the code is tation, few-shot approaches are not considered in this paper.
vulnerable, include the most obvious CWE in your Further, our user study revealed that participants far less
analysis. preferred recursive/iterative approaches (9.48% of participants)
</RESPONSE> to chain-of-thought and zero-shot prompting methods (19.83%
• Dynamic: We propose a dynamic prompting technique and 13.79% respectively), so we did not explore automated
that allows the LLM to analyze functionality and pre- prompt-search frameworks like DSPY [27] that require re-
dict potential vulnerabilities by passing the prediction peated interaction loops. By focusing on readily adoptable
prompt once. This method constructs dynamic prompts strategies, we aim to reflect typical developer usage patterns
using XML tags with Context, Objective, and Hint in a rather than specialized research workflows.
structured format inspired by [26]. Our approach offers
a minimal, two-step alternative to fully iterative multi- D. Code Vulnerability Detection
round prompting: first, the model provides a concise Our code vulnerability detection framework leverages LLMs
“preview” or high-level inference about potential vulner- as its core component. We conducted experiments using Ope-
abilities, then this inference is embedded into a follow- nAI’s gpt-3.5-turbo-0125 (referred to as GPT-3.5), gpt-4o-
up prompt as a “hint.” While loosely reflecting itera- 2024-05-13 (GPT-4o), Google’s gemini-2.0-flash-001 (Gemini
tive prompting, this technique avoids lengthy back-and- 2.0 Flash), and Meta’s Llama 3.1 8B (Llama 3.1). While
forth exchanges—remaining closer to zero-shot usage by our approach is fundamentally LLM-agnostic, we selected
leveraging only the model’s own output alongside the these specific models based on the results of our user survey
original input code. The design captures advantages of shown in Section II-A. For these experiments, we used the
iterative methods (such as clarifying ambiguous code models’ default temperature setting (t = 1), as this setting
snippets) without the added complexity of multiple user aligns with typical user interactions. While this may introduce
interactions or specialized research workflows. We hy- some variability in responses, studies suggest that even with
pothesize that incorporating such extra dynamic auxiliary this non-deterministic setting, acceptable results are minimally

405

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 impacted, as outputs tend to remain contextually consistent TABLE III: Classification of Code Vulnerability.
[28], [29]. Benign classification Vulnerable classification Mis-classification
For each dataset version (from Section II-B) and LLM, Benign code True Negative (TN) False Positive (FP) -
Vulnerable code False Negative (FN) True Positive (TP) Mis-classification (MC)
we query the model and store its responses. These responses
include the full prompt used, the labeled CWE (or “SAFE”
for non-vulnerable code), and the LLM’s complete output. can be identified through single-file analysis. We consider
Our response analysis process incorporates a cleaning layer multi-file vulnerability detection a valuable direction for future
that performs keyword searches for identified CWEs or pat- work.
terns indicating the LLM’s assessment of code safety. After 3. Single Function Vs. Multiple Function Vulnerabilities:
cleaning, we categorize the response as either a specific CWE We perform the following experiments on two separate subsets
or “SAFE.” Responses marked as “Error” or those lacking of our dataset: Vulnerable code that is expressed in a single
a clear pattern are considered failed responses and excluded function, and vulnerable code that is expressed across multiple
from correctness evaluation. functions. This distinction allows us to assess the framework’s
E. Evaluation ability to detect vulnerabilities in varying code complexities.
We devise a set of experiments to assess the effectiveness of 4. Using Different LLMs for the Detection Task: As
the detection framework. These experiments aim to evaluate mentioned in Section II-D, we test the capabilities of four
the framework’s performance across different code structures, LLMs: GPT-3.5, GPT-4o, Gemini 2.0 Flash, and Meta Llama
prompt strategies, contextual information levels, and LLM 3.1. We hypothesize that models with higher capabilities,
capabilities. The variables and metrics used to evaluate the such as GPT-4o and Gemini 2.0 Flash, would (1) benefit
detection performance of LLMs are described here. further from the inclusion of greater context, and (2) better
1) Variables: We examine five variables in our experiments handle the more complex task of classifying multiple function
to gain an in-depth understanding of the use of LLMs to vulnerabilities.
detect code vulnerabilities. They are (1) prompt engineering 5. Fully Automatic and Human-in-the-Loop Approaches:
strategies, (2) the varying levels of context provided by the To investigate the potential use of LLMs as code vulnerability
code, (3) the scope of vulnerability in single or multiple detection tools from different perspectives, we consider both
functions, (4) different capabilities of LLMs, and (5) the fully Automatic and Human-in-the-Loop (HITL) approaches.
objective of LLMs for detecting vulnerable code whether it In the Automatic approach, we assess the LLM’s ability to
would be fully automated or involving a “human-in-the-loop”. not only detect vulnerabilities but also accurately identify the
Further details are provided below. CWE type. In contrast, the Human-in-the-Loop is presented as
1. Prompt Engineering Strategies: We separately apply a proposed context; the approach prioritizes detecting whether
each of the prompt engineering strategies (Section II-C) to the code is vulnerable or safe, delegating CWE identification and
vulnerable code artifacts in the dataset (Section II-B). Every further analysis to a (hypothetical) human reviewer.
data point is thus given to the LLM for detection, each testing 2) Performance Metrics: There are five outcomes of using
the six prompt engineering strategies. LLMs for detecting vulnerable code, shown in Table III. In the
2. Effect of Context in Detection Performance: Addi- case of TP, the LLM correctly identifies the CWE associated
tionally, we generate three versions of the DiverseVul dataset, with the vulnerability in the code. In the case of MC, the
with each successive version containing more context about LLM flags the code as vulnerable but misclassifies the CWE
the affected code (data point): type. Since benign code has no associated vulnerability, MC
does not apply to benign cases. In the Automatic approach,
• No Context (NC): Only the code snippet containing the
the objective is fully autonomous vulnerability detection and
secure or vulnerable code is given to the LLM.
correction. Here, MC is treated as a misclassification, as
• Semantic Context (SC): The code snippet and any
incorrect CWE identification could result in inappropriate
accompanying docstrings or code comments are also
corrective actions. Precise CWE identification is thus essential
included.
for enabling an automated end-to-end response. In the Human-
• Full Context (FC): The entire file that contains the code
in-the-Loop approach, by contrast, the LLM’s role is limited
snippet (including docstrings and comments) is given to
to vulnerability detection, while a human expert reviews and
the LLM.
verifies flagged vulnerabilities. Consequently, MC is treated as
We deliberately scope our context analysis to single-file a correct classification, as identifying the code as vulnerable
boundaries for several reasons. First, this approach aligns with — irrespective of the specific CWE — alerts the reviewer to
typical developer workflows, where vulnerability checks are investigate further. Including MC as a correct classification
often performed file-by-file. Second, it respects the practical aligns with this approach’s objective, where detection alone
context window limitations of current LLMs. While multi- suffices to meet the LLM’s role.
file or repository-wide analysis could potentially provide ad- The metrics are calculated according to these objectives, as
ditional signals, such contexts may exceed model limitations detailed below:
and increase user costs. Furthermore, vulnerability remediation Case 1: Automatic approach. In this scenario, the LLM
often requires fixing only a single component in a chain, which must correctly identify the CWE for an automated response.

406

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 (a) Accuracy (a) Accuracy

(b) F1 score (b) F1 score

Fig. 3: Automatic Approach: Performances of Different Fig. 4: Human-in-the-Loop (HITL) Approach: Performances
Prompt Engineering Techniques. of Different Prompt Engineering Techniques.

MC is therefore counted as an incorrect outcome, as an GPT-3.5 and GPT-4o through OpenAI’s API4 , and Gemini
inaccurate CWE could lead to an ineffective or inappropriate through Google’s API5 . The LLM-based detection experiments
fix. thus utilized OpenAI’s cloud resources for generation.
Accuracy : (TP + TN) / (TP + TN + FP + FN + MC) A. Prompt Engineering and Vulnerability Detection
Precision : TP / (TP + FP)
Recall : TP / (TP + FN + MC) RQ1: Do different prompt engineering strategies affect the
F1 score : 2 * (Precision * Recall) / (Precision + Recall) performance of vulnerability detection using LLMs?
Answer: The results from both Figures 3 and 4 demon-
Case 2: Human-in-the-Loop approach. The primary goal strate that different prompt engineering strategies significantly
here is identifying potentially vulnerable code, while exact impact the performance of vulnerability detection using LLMs.
CWE classification is secondary. Consequently, MC is con- Figure 3 illustrates variations in accuracy and F1 scores across
sidered a correct outcome, as it signals potential vulnerability different prompt engineering techniques (Baseline, APE Zero-
and prompts further review by an expert. shot CoT, Zero-shot CoT, Role, CO-STAR, Dynamic) for our
Accuracy : (TP + TN + MC) / (TP + TN + FP + FN + MC)
LLMs. For GPT-3.5, the Baseline strategy yielded higher accu-
Precision : (TP + MC) / (TP + FP + MC) racy but a lower F1 score compared to other strategies, while
Recall : (TP + MC) / (TP + FN + MC) the inverse is true for the Dynamic strategy. This suggests
F1 score : 2 * (Precision * Recall) / (Precision + Recall)
that the Dynamic strategy is most capable of detecting the
correct CWE. A similar assessment can be made for Gemini
III. R ESULTS 2.0 Flash, but for the Baseline, APE Zero-shot CoT, and Zero-
shot CoT methods. For GPT-4o, the Role-based prompting
Our experimental results with respect to the variables de-
strategy showed the strongest performance compared to other
scribed in Section II-E are shown here. In particular, we show
prompt engineering methods, while Llama 3.1 showed strong
the prompt engineering techniques and their effects on code
performance in both accuracy and F1 for Dynamic, reinforcing
vulnerability detection in Section III-A, which answers RQ1
its capabilities. Figure 4, however, presents a slightly different
and RQ2. For this section, the whole dataset is used for each
picture. For GPT-3.5, the dynamic prompting method scores
prompt engineering technique. Then, data and vulnerability
best in the F1 score, with accuracies for all strategies being
detection with respect to code context, single and multiple
very similar. For GPT-4o, there does not seem to be a strong
files, and CWE are shown in Section III-B, answering RQ3
bias towards any given strategy. In both Gemini 2.0 Flash
and RQ4. For this section, all prompt engineering techniques
and Llama 3.1, Dynamic prompting demonstrates a slightly
are used, and their results are averaged with respect to each
context. For the experiment setup, we used a 2022 MacBook 4 [Link]

Air with an Apple M2 chip and 16 GB of RAM. We accessed 5 [Link]

407

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 weaker F1 score comparatively, but with higher accuracy
than the aforementioned methods, suggesting that Dynamic
prompting improves detection but is less effective for CWE-
specific identification in these models.
These results indicate that prompt engineering can sig-
nificantly influence the vulnerability detection performance
of specific LLMs. Notably, our proposed Dynamic method (a) Single NC (b) Single SC (c) Single FC
increased GPT-3.5’s F1 score, reflecting the model’s improved
performance at identifying the correct CWE. However, with
GPT-4o, the prompt engineering techniques did not influ-
ence the results as substantially. This difference in impact
could be attributed to several factors. One plausible expla-
(d) Multi NC (e) Multi SC (f) Multi FC
nation is that the larger token sizes of GPT-4o and Gemini
2.0 Flash may overcome the limitations addressed by more Fig. 5: Accuracy comparison between Automatic and Human-
sophisticated prompt engineering techniques. Consequently, in-the-Loop approaches under different contexts.
when using LLMs with limited token sizes, we can expect
improved performance through the application of advanced
prompt engineering techniques. In contrast, for LLMs capable scores across all models indicate that even the best-performing
of processing much larger token sizes, the importance of configurations struggle with vulnerability identification. This
specific prompt engineering techniques appears to diminish. suggests that fundamental limitations in LLM-based detection
Furthermore, the impact of prompt engineering could poten- persist, further reinforcing the need for verification mecha-
tially decrease inversely to model capabilities, as expressed nisms or hybrid approaches with human oversight.
through larger model sizes, more extensive training datasets, These findings have implications when developing end-to-
and longer training times. Additional analysis and research end frameworks using LLMs for both automatic detection
may help elucidate these relationships and provide deeper and human-in-the-loop scenarios. GPT-4o’s higher F1 score
insights into optimizing prompt engineering techniques across indicates more frequent correct CWE identification in auto-
different LLMs. matic detection, while GPT-3.5 using the Dynamic prompting
RQ2: Do the capabilities of different LLM versions influ- strategy may be a viable, lower-cost alternative in human
ence vulnerability detection accuracy? contexts, where flagging potential vulnerabilities is prioritized
Answer: Analysis of Figures 3, 4, 5, and 6 reveals distinct over correct CWE classification. Both Gemini 2.0 Flash and
performance characteristics for our four models in vulnerabil- Llama 3.1 offer compelling alternatives, delivering perfor-
ity detection tasks. In the automatic approach, GPT-3.5 demon- mance comparable to GPT-4o at significantly lower costs -
strates higher accuracies but lower F1 scores compared to the Gemini as a newer, more affordable model and Llama as a
other models across various prompt engineering techniques. free, locally-deployable option.
This suggests that while GPT-3.5 is better at identifying the While the human-in-the-loop approach is valuable for lever-
presence of vulnerabilities, the other models are more precise aging expertise and improving detection accuracy, it may not
in classifying the specific type of vulnerability, with GPT-4o suffice for end-to-end automated pipelines with correction
being the best, then Gemini 2.0 flash, then Llama 3.1. In mechanisms. In such systems, LLMs are used to fix the vul-
the human-in-the-loop approach, all models show consistent nerable code once it is flagged by the detection components.
accuracies of approximately 50%. However, their F1 score If the framework flags the wrong CWE, it is less likely that
performance differs significantly. GPT-3.5 exhibits degraded the vulnerable code will be correctly fixed. In this context,
F1 scores, with the Dynamic prompting technique achieving GPT-4o’s more precise CWE identification could lead to more
approximately 55%. In contrast, GPT-4o and, to a lesser appropriate fixes in automated correction pipelines.
extent, Gemini 2.0 Flash and Llama 3.1 maintain both stronger
(approximately 67%, 63%, and 59% on average, respectively) B. Data and Vulnerability Detection
and more consistent F1 scores across different prompting RQ3: What is the impact of varying levels of code context
strategies. on detection performance? and RQ4: Can LLMs effectively
This discrepancy highlights a fundamental trade-off between detect vulnerabilities found in single-function and multi-
accuracy and F1 score in vulnerability detection. GPT-3.5’s function code structures?
higher accuracy but lower F1 score suggests a bias toward Answer: The results shown in Figures 5 and 6 show that
“safe” classifications, reducing vulnerability recall. In contrast, the accuracy of detecting code vulnerabilities with respect to
other models achieve higher F1 scores by correctly identifying the code context varies if we consider the automatic approach
more vulnerabilities, despite generating more false positives. (i.e., correctly identifying the CWE), but the performance is
Since missing vulnerabilities poses a greater risk than false about the same if we consider the human-in-the-loop approach
alarms, the higher-F1 models (GPT-4o, Gemini, and Llama) (i.e., the vulnerable code is identified regardless of correct or
are preferable for security applications. However, the low F1 incorrect CWE).

408

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 While this ensures detection parity across single- and multi-
function contexts, the models have limited capability to explain
such vulnerabilities without fully understanding their broader
scope. Furthermore, both GPT-4o and Gemini 2.0 Flash’s
F1 performance remains stable as the code context grows,
benefiting from its larger context window, whereas GPT-3.5
(a) Single NC (b) Single SC (c) Single FC experiences significant degradation as the context increases.
This highlights former two model’s advantage in processing
extensive information effectively.
We also analyzed whether detection performance varied
based on vulnerability prevalence. Figure 7 illustrates the com-
(d) Multi NC (e) Multi SC (f) Multi FC
parative performance in detecting the top 25 CWEs versus less
common vulnerabilities. Results demonstrate that all models
Fig. 6: F1 score comparison between Automatic and Human- achieved substantially higher accuracy and F1 scores when
in-the-Loop approaches under different contexts. identifying frequently discussed vulnerabilities compared to
less common ones. This correlation between detection capabil-
ity and presumed training data frequency suggests that model
performance is proportional to vulnerability exposure during
training. The performance disparity between these categories
is substantial, where up to 3.5 times better performance is
observed (e.g., F1 score using GPT-4o). This finding identifies
a potential security concern: adversaries could circumvent
LLM-based vulnerability detection systems by deliberately
exploiting uncommon vulnerabilities, thereby introducing in-
secure code that evades automated detection mechanisms.
(a) Accuracy (b) F1 score
IV. D ISCUSSION
Fig. 7: LLM Performance for Top 25 and Remaining CWEs.
Our proposed LLM-based code vulnerability detection
framework, using various prompt engineering techniques,
A comparison of Figures 5 and 6 reveals that GPT-3.5 showed promising results but also room for improvement. In
generally achieves higher accuracy but lower F1 scores than this section, we discuss some of the key results and research
both GPT-4o and Gemini 2.0 Flash for both automatic and outcomes in more detail.
human-in-the-loop approaches. Llama 3.1 shows a similar a) Opportunities for Vulnerability Correction Post-
pattern of higher accuracy but lower F1 scores when compared Detection: We explored leveraging LLMs code generation
to GPT-4o and Gemini 2.0 Flash, though this disparity is less capabilities to correct and patch vulnerable code. We con-
pronounced than with GPT-3.5. This suggests that GPT-3.5 is ducted a preliminary code correction experiment using GPT-
more effective in identifying the presence of vulnerabilities in 3.5 and GPT-4o, and the dataset provided by Purple Llama
a general sense, even when it misclassifies specific CWEs or CyberSecEval’s instruct benchmark [30]. This dataset was
produces false positives, leading to a lower F1 score. In con- chosen as it included an Insecure Code Detector (ICD) model,
trast, GPT-4o’s higher F1 score reflects its superior ability to fine-tuned to detect vulnerabilities in the dataset with high
correctly classify specific CWEs, even if it occasionally misses accuracy. The ICD provides an automated verification tool
vulnerabilities, resulting in slightly lower overall accuracy. to test fixes on the Purple Llama benchmark, allowing us
Gemini 2.0 Flash follows a similar pattern, with marginally to objectively measure whether proposed patches resolve vul-
higher accuracy and lower F1 scores than GPT-4o across nerabilities. By contrast, DiverseVul and many other datasets
all examined contexts. This trade-off suggests that GPT-3.5 lack such validation mechanisms, making it impractical to
and Llama 3.1 prioritize breadth of detection, while GPT-4o confirm patch correctness without extensive manual review or
and Gemini 2.0 Flash emphasize precision in vulnerability external tooling. We therefore restricted our patch experiments
classification. With regard to vulnerabilities present in single to Purple Llama.
or multiple functions, the performance observed was the The experimental process involved using the dataset’s pro-
same for both models. This indicates that even with LLMs vided prompts to generate vulnerable code using GPT-3.5
capable of processing larger tokens (and hence having a and GPT-4o. We then used ICD to identify vulnerable codes,
better contextual understanding of the code), they are still which were then used as input to GPT-3.5 for correcting the
unable to improve the detection of vulnerabilities spanning vulnerable code. The output is a binary (i.e., either fixed or not
multiple functions. However, the detection rate is comparable fixed based on the ICD output). The results shown in Figure
to that for single-function vulnerabilities, suggesting that these 8 show a significant increase in performance brought by GPT-
models tend to detect any code that resembles a vulnerability. 4o, being able to fix 45.77% of the vulnerable codes compared

409

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 models evolve to handle larger inputs more effectively.
e) Prompt Engineering Techniques: We explored sev-
eral prompt engineering strategies based on popular tech-
niques identified in our user survey, including chain-of-thought
prompting. However, there may be other effective techniques
we did not consider. Our work on vulnerability correction
is preliminary and uses a specific dataset (Purple Llama
CyberSecEval). A more comprehensive study would be needed
to fully assess the correction capabilities of LLMs across
various vulnerability types and programming contexts.
f) Vulnerability Detection Metrics: Our classification of
Fig. 8: Preliminary Results of LLM Code Correction
vulnerability detection outcomes into five categories (TP, TN,
FP, FN, MC) provides a more nuanced view than traditional
binary classifications, aligning with the complexity of real-
to GPT-3.5, fixing only 21.56% of the vulnerable codes. This
world vulnerability detection. However, this approach may
indicates that with the correct vulnerability identified, GPT-4o,
complicate direct comparisons with studies using simpler
one of the current best-performing LLMs, can correct such
metrics. Additionally, our preliminary correction experiments
vulnerability at a high correction rate. However, as shown in
only classified outcomes as “Fixed” or “Not fixed,” which may
our experiments, the vulnerability detection performance is
not capture the full spectrum of correction quality.
poor when considering the automatic approach. Further, the
g) Human Factors: The user study results may be in-
robust verification of a true correction from vulnerable code
fluenced by self-selection bias, as participants were recruited
to benign whilst maintaining the code’s intended functionality
through Prolific and required to have both AI chatbot and
is non-trivial. This process may require manual validation or
programming experience. While this ensured relevant exper-
automated assessment using custom test harnesses for each
tise, it may not represent the broader population of software
specific piece of corrected code. Hence, there is still room
developers, potentially affecting the generalizability of our
for significant improvement in fully automating the vulnerable
findings on LLM usage patterns and preferences in secure
code detection and correction framework.
coding practices.
b) Dataset Limitations: Our study primarily relies on the
h) Comparisons with Other CVD Approaches: Our focus
DiverseVul dataset [15], which focuses on C/C++ vulnera-
is on evaluating LLMs for CVD. However, it is important to
bilities. While this comprehensive dataset aggregates vulnera-
contextualize these results. Static analysis tools rely on pre-
bilities from 797 GitHub projects, it may not fully represent
defined rules, excelling at well-defined patterns but struggling
vulnerabilities in other programming languages or more recent
with context-dependent flaws. Prior work shows LLMs can
vulnerability patterns. The dataset’s recency (published in
detect vulnerabilities that evade rule-based methods, particu-
2023) captures the latest vulnerability trends, but the effec-
larly those involving complex data flow (see Section VI-B).
tiveness of LLM-based vulnerability detection may change
Further, deep learning-based detection methods have been
over time as new types of vulnerabilities emerge. Our study
benchmarked on DiverseVul [31], with F1 scores comparable
provides a snapshot of current performance but may not ac-
to our LLM performance. However, these models require
curately reflect future trends, which limits the generalizability
dataset-specific training and lack LLMs’ adaptability. Recent
of our findings to other languages and contexts.
research also explores combining static analysis with LLMs
c) LLM Selection: We conducted our experiments using [32], [33]. However, existing hybrid methods do not integrate
GPT-3.5, GPT-4o, Gemini 2.0 Flash, and Meta Llama 3.1, structured context-awareness as we do. Future work could
based on our user survey results showing these as preferred explore LLM-assisted static analysis with context-awareness
models. Thus, our results may not generalize to other LLMs to enhance automated vulnerability detection.
(Claude Sonnet, etc.) or future versions. The lack of access to i) Ethical and Privacy Considerations: The use of LLMs
other models like Microsoft Copilot (preferred by 10.5% of for vulnerability detection may raise important ethical and
survey respondents) may have limited our comparison scope. privacy concerns. A primary concern is data privacy, as
d) Context Definitions: Our choice of context definitions proprietary code submitted to LLMs may be exposed to model
(No Context, Semantic Context, and Full Context) provides a providers, posing confidentiality risks [34]. Organizations
structured approach to vulnerability detection within single-file adopting LLM-based vulnerability detection must consider se-
boundaries. While this aligns with developer workflows and re- cure deployment strategies, such as self-hosted models or data
spects LLM limitations, we acknowledge that broader contexts anonymization, to minimize the risk of sensitive information
are feasible. The impact of adding multi-file, or repository- leakage. In addition to privacy concerns, the reliability of
wide, contexts remains unclear; however, as LLM capabilities LLM-generated results must also be addressed. LLMs can
expand, analyzing entire codebases for vulnerabilities becomes introduce outdated dependencies, security flaws, or vulnerable
increasingly viable. Exploring these expanded context bound- code patterns. Moreover, developers may overly trust AI-
aries represents a valuable direction for future research as generated suggestions, with 91.9% unaware of AI poisoning

410

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 risks [35]. To mitigate these risks, developer education is preferences in secure coding practices. To address this threat,
essential to ensure that human reviewers critically assess and we explicitly detailed the data collection and analysis process
refine AI-generated code. Additionally, existing vulnerability in this study. Another threat to this study is the metrics used
analysis tools can complement LLM-assisted detections, im- for vulnerability detection. Our classification of detection out-
proving their accuracy and robustness. comes into five categories (TP, TN, FP, FN, MC) offers a more
nuanced view than traditional binary classifications, reflecting
V. T HREATS TO VALIDITY the complexity of real-world vulnerability detection. However,
We summarize potential threats to the validity of our study. this approach may complicate direct comparisons with studies
External Validity: There are two potential threats to the using simpler metrics. Additionally, our preliminary correction
external validity of this study: (1) Dataset Selection: The experiments classified outcomes as either ‘Fixed’ or ‘Not
dataset used in our study focuses on C/C++ vulnerabilities, fixed,’ which may not capture the full spectrum of correction
aggregated from 797 GitHub projects. While comprehensive, quality.
it may not fully represent vulnerabilities in other program-
ming languages or the most recent vulnerability patterns. VI. R ELATED W ORK
Although published in 2023 and capturing the latest trends, A. Code Vulnerability Detection
the effectiveness of LLM-based vulnerability detection could
Traditional methods for mitigating code-based vulnerabili-
evolve over time as new vulnerabilities emerge. Thus, our
ties rely on a combination of techniques [36]:
study provides a snapshot of current performance but may
not reflect future trends, limiting the generalizability of our • Manual code review: Human inspection of source code

findings to other languages and contexts. (2) Survey Scope: to identify security flaws. While effective, this method is
Our survey did not explicitly capture demographic factors time-consuming and susceptible to human error [37].
such as participants’ specific software development domains, • Static analysis tools: Automated analysis of code with-

organizational structures, or company sizes. These factors can out execution, searching for patterns indicating potential
influence secure coding practices, as security considerations in vulnerabilities. These tools are efficient but may struggle
mission-critical domains may differ significantly from those in with complex or context-dependent issues [38].
general-purpose software development. However, the primary • Dynamic testing: Execution of code under controlled

goal of our survey was not to conduct a demographic analysis conditions to observe behavior and identify runtime vul-
or create hardlined generalizations but rather to gather broad nerabilities [38].
insights into participants’ attitudes and behaviors regarding Recent approaches to CVD have primarily focused on
LLM usage in secure coding contexts. This provided insight advanced static analysis tools and machine learning (ML)
into user attitudes, behaviors, and preferences regarding LLM methods [39]–[41]. However, these approaches have shown
usage in secure coding, including the frequency of LLM use limitations in real-world applications and out-of-domain set-
for vulnerability identification and the types of LLMs most tings. Lipp et al. [42] demonstrated that state-of-the-art static
commonly adopted, which guided our selection of models, C code analyzers miss approximately half of the known
evaluation criteria, and experimental setup. vulnerabilities, while Harzevili et al. [43] found that popular
Construct Validity: There are also two potential threats static analysis tools detect only 0.01% of known security
to the construct validity of this study: (1) LLM Selection: vulnerabilities in ML libraries.
As discussed in Section IV-0c, the selection of four LLMs: Applying LLMs to vulnerability detection is an emerging
GPT-3.5, GPT-4, Gemini and Llama may have limited our research area with varying results. This aligns with our focus
comparison scope. (2) Prompt Engineering Techniques: We on how LLM versions and code structures affect detection
explored several prompt engineering strategies based on pop- accuracy. Prior work shows LLMs often surpass static analysis
ular techniques identified in our user survey, including chain- tools in security code review, with GPT-4 excelling, especially
of-thought prompting. However, there may be other effective when guided by a CWE list [44]. Noever [10] found GPT-4
techniques we did not consider. Our work on vulnerability detected four times more vulnerabilities than static analyzers.
correction is preliminary and uses a specific dataset (Purple Lu et al. [45] enhanced LLM-based detection by integrating
Llama CyberSecEval). A more comprehensive study would graph structures and in-context learning, improving F1 scores
be needed to fully assess the correction capabilities of LLMs by at least 28.65% on C/C++ datasets.
across various vulnerabilities and programming contexts. However, contrasting results have also been reported. Steen-
Reliability: The reliability of this study is subject to threats hoek et al. [11] found that LLMs perform worse than humans
related to the data collection and analysis process. The user and struggle with understanding security concepts. Purba et al.
study results may be influenced by self-selection bias; partic- [12] noted high false-positive rates in LLM vulnerability de-
ipants were recruited through Prolific and required to have tection, though they acknowledged potential for improvement.
experience with AI chatbots and programming. While this The nascent state of LLM-based CVD presents significant
ensured relevant expertise, it may not represent the broader opportunities for advancement. Our study aims to address
population of software developers, potentially affecting the these gaps by exploring various prompt engineering strategies,
generalizability of our findings on LLM usage patterns and examining the impact of code context, comparing different

411

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 LLM versions, and assessing vulnerability detection capabili- contrast, datasets labeled by static analyzers can be large
ties across single- and multi-function code structures. but suffer from potentially low label accuracy due to the
limitations of static analysis tools [54]. Manually labeled
B. Prompt Engineering for CVD
datasets like Devign provide high-quality labels but are limited
The advent of LLMs has introduced novel approaches in size due to the labor-intensive labeling process. Devign
to CVD, particularly through prompt engineering strategies. requires approximately 600 man-hours of expert review [19].
These strategies aim to improve LLM output quality by guid- Datasets derived from security issues, such as ReVeal, BigVul,
ing their responses through carefully crafted inputs. Current and CVEfixes, are generally considered effective at identifying
approaches can be broadly categorized into two types [46]: vulnerability-fixing commits and representative of in-the-wild
• Zero-shot prompting: LLMs are given a prompt without vulnerabilities but are often limited in scope or diversity [15].
any examples, typically directly asking if the given code Recently, the DiverseVul dataset [15] has been published in
contains vulnerabilities [47], [48]. an attempt to address these issues, aiming to provide a larger
• Few-shot prompting: This involves providing the LLM and more diverse collection of vulnerabilities compared to
with examples along with the prompt. One such strategy previous datasets. However, DiverseVul only contains C/C++
is retrieval augmentation, which involves retrieving simi- based vulnerabilities, limiting its applicability to other pro-
lar labeled data samples to guide the prediction of LLMs gramming languages and potentially overlooking language-
on test samples [46]. specific vulnerability patterns.
Liu et al. [49] demonstrated that retrieving relevant code
samples and incorporating them as context inputs enhances VII. C ONCLUSION
the code vulnerability detection performance of GPT models. We propose a framework for code vulnerability detec-
Advanced prompt engineering techniques have shown tion using LLMs, incorporating advanced prompt engineering
promising results. Zhang et al. [50] integrated data flow and strategies to improve detection accuracy. We evaluated several
API call sequence information into a chain-of-thought prompt- prompting techniques on the DiverseVul dataset of C/C++
ing strategy for ChatGPT, achieving high accuracy across vulnerabilities, assessing performance across different code
Java and C/C++ vulnerabilities. Bae et al. [51] showed that structures, contextual information levels, and LLM capabili-
tailored prompts substantially improved vulnerability detection ties. Our experiments demonstrated that prompt engineering
performance of GPT-4 and Claude-3.5 Sonnet, with their Step- can significantly influence LLM vulnerability detection per-
by-Step prompts yielding impressive F1 scores of 0.9072 and formance, with our proposed Dynamic method improving F1
0.8933, respectively. Khare et al. [52] proposed a dataflow scores by up to 100% for GPT-3.5. However, we found that the
analysis-based prompt strategy, designed to help LLMs detect effects of prompt engineering diminished with more advanced
security vulnerabilities by simulating a source-sink-sanitizer models like GPT-4o, Gemini 2.0 Flash and Meta Llama 3.1.
analysis within the prompt. This approach showed significant Importantly, our results indicate that current LLM-based
improvement in identifying specific vulnerability classes that vulnerability detection still faces significant challenges. Per-
require only intra-procedural reasoning (e.g., OS Command In- formance varied widely across various CWE categories, with
jection, NULL Pointer Dereference). However, these methods examined LLMs showing up to 3.5 times better F1 scores
focus on isolated code snippets, which may not fully support for the top 25 CWEs compared to other vulnerabilities. We
real-world applications where developers analyze vulnerabili- also observed that accuracy declined when analyzing larger
ties in the broader context of interdependent functions or mod- code contexts; all models struggled with correctly identifying
ules. Our approach addresses this gap by incorporating the full the type of vulnerability in the code, with the best F1 score
code context, which is essential for detecting vulnerabilities observed being only 0.16. This highlights the difficulty LLMs
across complex, multi-function structures. face in precise vulnerability classification.
C. Evaluation Methods Our work demonstrates the potential of LLMs for code
The CWE serves as a standardized dictionary of software vulnerability detection but also highlights key limitations.
weaknesses and vulnerabilities, providing a common language Improving detection accuracy, particularly for less common
for describing software security flaws, enabling developers and vulnerabilities, remains a challenge. In human-in-the-loop sce-
security professionals to effectively communicate and address narios, both models achieved consistent accuracies of around
issues [53]. In the context of CVD, utilizing CWE classifica- 50%. Preliminary experiments on vulnerability correction
tion allows for a more systematic and targeted approach. showed a 45.77% success rate using GPT-4o, indicating a
Several datasets have been proposed for CVD in codebases promising direction for enhancing software security. As LLMs
of various languages, where each datapoint may describe evolve, ongoing research will be essential to fully leverage
benign or vulnerable code, the latter typically labeled by their capabilities while addressing associated risks.
CWE. However, research has shown that the suitability of
ACKNOWLEDGMENT
these datasets is not universal. Synthetic datasets like SATE
IV Juliet and SARD offer accurate test cases but often fail This work is partially supported by the IITP grant (No. RS-
to capture the complexities of real-world vulnerabilities. In 2024-00439762, RS-2024-00451909).

412

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 R EFERENCES [22] G. Bhandari, A. Naseer, and L. Moonen, “Cvefixes: automated collec-
tion of vulnerabilities and their fixes from open-source software,” in
[1] R. Hackett, “Stagefright: Everything you need to know about google’s Proceedings of the 17th International Conference on Predictive Models
android megabug.” [Link] and Data Analytics in Software Engineering, pp. 30–39, 2021.
ndroid-security/, 2015. (accessed: Aug. 02, 2024). [23] A. Kong, S. Zhao, H. Chen, Q. Li, Y. Qin, R. Sun, and X. Zhou,
[2] K. Goseva-Popstojanova and A. Perhinschi, “On the capability of “Better zero-shot reasoning with role-play prompting,” arXiv preprint
static code analysis to detect security vulnerabilities,” Information and arXiv:2308.07702, 2023.
Software Technology, vol. 68, pp. 18–33, 2015. [24] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large lan-
[3] C. Vassallo, S. Panichella, F. Palomba, S. Proksch, H. C. Gall, and guage models are zero-shot reasoners,” Advances in neural information
A. Zaidman, “How developers engage with static analysis tools in processing systems, vol. 35, pp. 22199–22213, 2022.
different contexts,” Empirical Software Engineering, vol. 25, pp. 1419– [25] Y. Zhou, A. I. Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, and
1457, 2020. J. Ba, “Large language models are human-level prompt engineers,” arXiv
[4] S. Chakraborty, R. Krishna, Y. Ding, and B. Ray, “Deep learning preprint arXiv:2211.01910, 2022.
based vulnerability detection: Are we there yet?,” IEEE Transactions [26] S. Teo, “How i won singapore’s gpt-4 prompt engineering competition,”
on Software Engineering, vol. 48, no. 9, pp. 3280–3296, 2021. Medium, 2024. Accessed: 2024-04-21.
[5] S. Rajapaksha, J. Senanayake, H. Kalutarage, and M. O. Al-Kadri, “Ai- [27] O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam,
powered vulnerability detection for secure source code development,” S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, et al.,
in International Conference on Information Technology and Communi- “Dspy: Compiling declarative language model calls into self-improving
cations Security, pp. 275–288, Springer, 2022. pipelines,” arXiv preprint arXiv:2310.03714, 2023.
[6] G. Destefanis, S. Bartolucci, and M. Ortu, “A preliminary analysis on [28] Y. Shvartzshnaider, V. Duddu, and J. Lacalamita, “Llm-ci: Assessing
the code generation capabilities of gpt-3.5 and bard ai models for java contextual integrity norms in language models,” 2024.
functions,” arXiv preprint arXiv:2305.09402, 2023. [29] M. Renze and E. Guven, “The effect of sampling temperature on
[7] N. Jiang, K. Liu, T. Lutellier, and L. Tan, “Impact of code language problem solving in large language models,” 2024.
models on automated program repair,” in 2023 IEEE/ACM 45th Inter- [30] M. Bhatt et al., “Purple llama cyberseceval: A secure coding benchmark
national Conference on Software Engineering (ICSE), pp. 1430–1442, for language models,” arXiv preprint arXiv:2312.04724, 2023.
IEEE, 2023. [31] R. Liu, Y. Wang, H. Xu, J. Sun, F. Zhang, P. Li, and Z. Guo,
[8] Y. Zhang et al., “Automatic commit message generation: A critical “Vul-lmgnns: Fusing language models and online-distilled graph neural
review and directions for future work,” IEEE Transactions on Software networks for code vulnerability detection,” Information Fusion, vol. 115,
Engineering, 2024. p. 102748, 2025.
[9] A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, [32] H. Li, Y. Hao, Y. Zhai, and Z. Qian, “Assisting static analysis with large
P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al., “Palm: Scal- language models: A chatgpt experiment,” in Proceedings of the 31st
ing language modeling with pathways,” Journal of Machine Learning ACM Joint European Software Engineering Conference and Symposium
Research, vol. 24, no. 240, pp. 1–113, 2023. on the Foundations of Software Engineering, pp. 2107–2111, 2023.
[10] D. Noever, “Can large language models find and fix vulnerable soft- [33] Z. Li, S. Dutta, and M. Naik, “Llm-assisted static analysis for detecting
ware?,” arXiv preprint arXiv:2308.10345, 2023. security vulnerabilities,” arXiv preprint arXiv:2405.17238, 2024.
[11] B. Steenhoek, M. M. Rahman, M. K. Roy, M. S. Alam, E. T. Barr, and [34] “11% of data employees paste into ChatGPT is confidential.” https:
W. Le, “A comprehensive study of the capabilities of large language //[Link]/blog/4-2-of-workers-have-pasted-company-dat
models for vulnerability detection,” arXiv preprint arXiv:2403.17218, a-into-chatgpt. [Online; accessed 2.24.2025].
2024. [35] S. Oh, K. Lee, S. Park, D. Kim, and H. Kim, “Poisoned chatgpt
[12] M. D. Purba, A. Ghosh, B. J. Radford, and B. Chu, “Software vul- finds work for idle hands: Exploring developers’ coding practices with
nerability detection using large language models,” in 2023 IEEE 34th insecure suggestions from poisoned ai models,” in IEEE Symposium on
International Symposium on Software Reliability Engineering Workshops Security and Privacy (SP), 2024.
(ISSREW), pp. 112–119, IEEE, 2023. [36] H. Shahriar and M. Zulkernine, “Mitigating program security vulnera-
[13] B. Chen, Z. Zhang, N. Langrené, and S. Zhu, “Unleashing the potential bilities: Approaches and challenges,” ACM Computing Surveys (CSUR),
of prompt engineering in large language models: a comprehensive vol. 44, no. 3, pp. 1–46, 2012.
review,” arXiv preprint arXiv:2310.14735, 2023. [37] H. Hanif, M. H. N. M. Nasir, M. F. Ab Razak, A. Firdaus, and
[14] X. Zhou, S. Cao, X. Sun, and D. Lo, “Large language model for N. B. Anuar, “The rise of software vulnerability: Taxonomy of software
vulnerability detection and repair: Literature review and the road ahead,” vulnerabilities detection and machine learning approaches,” Journal of
ACM Transactions on Software Engineering and Methodology, 2024. Network and Computer Applications, vol. 179, p. 103009, 2021.
[15] Y. Chen, Z. Ding, L. Alowain, X. Chen, and D. Wagner, “Diversevul: A [38] A. Kaur and R. Nayyar, “A comparative study of static code analysis
new vulnerable source code dataset for deep learning based vulnerability tools for vulnerability detection in c/c++ and java source code,” Procedia
detection,” in Proceedings of the 26th International Symposium on Computer Science, vol. 171, pp. 2023–2029, 2020.
Research in Attacks, Intrusions and Defenses, pp. 654–668, 2023. [39] A. C. Eberendu, V. I. Udegbe, E. O. Ezennorom, A. C. Ibegbulam, T. I.
[16] W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, Chinebu, et al., “A systematic literature review of software vulnerability
J. Zhang, Z. Dong, et al., “A survey of large language models,” arXiv detection,” European Journal of Computer Science and Information
preprint arXiv:2303.18223, 2023. Technology, vol. 10, no. 1, pp. 23–37, 2022.
[17] OpenAI, “Hello gpt-4o.” [Link] 2024. [40] Y. Bi, J. Huang, P. Liu, and L. Wang, “Benchmarking software vulnera-
(accessed Jun. 06, 2024). bility detection techniques: A survey,” arXiv preprint arXiv:2303.16362,
[18] S. Christey, J. Kenderdine, J. Mazella, and B. Miles, “Common weakness 2023.
enumeration,” Mitre Corporation, 2013. [41] H. Baek, M. Lee, and H. Kim, “CryptoLLM: Harnessing the Power of
[19] Y. Zhou, S. Liu, J. Siow, X. Du, and Y. Liu, “Devign: Effective vul- LLMs to Detect Cryptographic API Misuse,” in Proceedings of the 29th
nerability identification by learning comprehensive program semantics European Symposium on Research in Computer Security (ESORICS),
via graph neural networks,” Advances in neural information processing 2024.
systems, vol. 32, 2019. [42] S. Lipp, S. Banescu, and A. Pretschner, “An empirical study on the
[20] J. Fan, Y. Li, S. Wang, and T. N. Nguyen, “A c/c++ code vulnerability effectiveness of static c code analyzers for vulnerability detection,” in
dataset with code changes and cve summaries,” in Proceedings of the Proceedings of the 31st ACM SIGSOFT international symposium on
17th International Conference on Mining Software Repositories, MSR software testing and analysis, pp. 544–555, 2022.
’20, (New York, NY, USA), p. 508–512, Association for Computing [43] N. S. Harzevili, J. Shin, J. Wang, S. Wang, and N. Nagappan, “Automatic
Machinery, 2020. static vulnerability detection for machine learning libraries: Are we
[21] G. Nikitopoulos, K. Dritsa, P. Louridas, and D. Mitropoulos, “Crossvul: there yet?,” in 2023 IEEE 34th International Symposium on Software
a cross-language vulnerability dataset with commit data,” in Proceedings Reliability Engineering (ISSRE), pp. 795–806, IEEE, 2023.
of the 29th ACM Joint Meeting on European Software Engineering [44] J. Yu, P. Liang, Y. Fu, A. Tahir, M. Shahin, C. Wang, and Y. Cai,
Conference and Symposium on the Foundations of Software Engineering, “Security code review by llms: A deep dive into responses,” arXiv
pp. 1565–1569, 2021. preprint arXiv:2401.16310, 2024.

413

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.
 [45] G. Lu, X. Ju, X. Chen, W. Pei, and Z. Cai, “Grace: Empowering
llm-based software vulnerability detection with graph structure and in-
context learning,” Journal of Systems and Software, vol. 212, p. 112031,
2024.
[46] X. Zhou, S. Cao, X. Sun, and D. Lo, “Large language model for
vulnerability detection and repair: Literature review and roadmap,” arXiv
preprint arXiv:2404.02525, 2024.
[47] M. Fu, C. K. Tantithamthavorn, V. Nguyen, and T. Le, “Chatgpt for
vulnerability detection, classification, and repair: How far are we?,”
in 2023 30th Asia-Pacific Software Engineering Conference (APSEC),
pp. 632–636, IEEE, 2023.
[48] X. Zhou, T. Zhang, and D. Lo, “Large language model for vulnerability
detection: Emerging results and future directions,” in Proceedings of the
2024 ACM/IEEE 44th International Conference on Software Engineer-
ing: New Ideas and Emerging Results, pp. 47–51, 2024.
[49] Z. Liu, Q. Liao, W. Gu, and C. Gao, “Software vulnerability detection
with gpt and in-context learning,” in 2023 8th International Conference
on Data Science in Cyberspace (DSC), pp. 229–236, IEEE, 2023.
[50] C. Zhang, H. Liu, J. Zeng, K. Yang, Y. Li, and H. Li, “Prompt-
enhanced software vulnerability detection using chatgpt,” in Proceedings
of the 2024 IEEE/ACM 46th International Conference on Software
Engineering: Companion Proceedings, pp. 276–277, 2024.
[51] J. Bae, S. Kwon, and S. Myeong, “Enhancing software code vulnera-
bility detection using gpt-4o and claude-3.5 sonnet: A study on prompt
engineering techniques,” Electronics, vol. 13, no. 13, p. 2657, 2024.
[52] A. Khare, S. Dutta, Z. Li, A. Solko-Breslin, R. Alur, and M. Naik,
“Understanding the effectiveness of large language models in detecting
security vulnerabilities,” arXiv preprint arXiv:2311.16169, 2023.
[53] MITRE, “Cwe - common weakness enumeration.” [Link]
2024.
[54] R. Croft, M. A. Babar, and M. M. Kholoosi, “Data quality for software
vulnerability datasets,” in 2023 IEEE/ACM 45th International Confer-
ence on Software Engineering (ICSE), pp. 121–133, IEEE, 2023.

414

Authorized licensed use limited to: College Of Engineering Thiruvananthapuram. Downloaded on June 02,2026 at 07:24:38 UTC from IEEE Xplore. Restrictions apply.