Towards Detecting LLMs Hallucination via

Markov Chain-based Multi-agent Debate Framework

Xiaoxi Sun1∗, Jinpeng Li1∗, Yan Zhong1 , Dongyan Zhao1†, Rui Yan2†
1
Wangxuan Institute of Computer Technology, Peking University
2
Gaoling School of Artifical Intelligence, Renmin University of China
{sunxiaoxi, zhongyan}@[Link], [Link]@[Link],
zhaody@[Link], ruiyan@[Link]

Claim Detection Multi-agent Verification

Abstract Do you think this claim is
Response: factual based on evidence?
Eise Eisinga was a Dutch mathematician and
The advent of large language models (LLMs) astronomer who is known for building ...
I think...
has facilitated the development of natural lan-
I agree...
guage text generation. It also poses unprece-
arXiv:2406.03075v1 [[Link]] 5 Jun 2024

dented challenges, with content hallucination Claims: I disagree...

Eise Eisinga was a Dutch mathematician.
emerging as a significant concern. Existing Eise Eisinga was an astronomer ... Consider both agents'
opinion, I think...
solutions often involve expensive and complex
interventions during the training process. More-
over, some approaches emphasize problem dis- Evidence Retrieval
assembly while neglecting the crucial valida- Claim: Eise Eisinga was an astronomer Evidences:
1. Eise Eisinga: Amateur astronomer ...
tion process, leading to performance degrada- [Link] Jeltes Eisinga was a Frisian
tion or limited applications. To overcome these amateur astronomer who built the Eise
Eisinga Planetarium in his house in
limitations, we propose a Markov Chain-based Franeker, Dutch Republic. The orrery
still exists and is the oldest
Queries: Who was Eise Eisinga? functioning planetarium ...
multi-agent debate verification framework to Eise Eisinga astronomer

enhance hallucination detection accuracy in

concise claims. Our method integrates the fact- Figure 1: Overview of the fact-checking process, which
checking process, including claim detection, includes three distinct stages. Claim Detection, we
evidence retrieval, and multi-agent verification. leverage large language models such as ChatGPT to
In the verification stage, we deploy multiple obtain different claims. Evidence Retrieval, we prompt
agents through flexible Markov Chain-based ChatGPT to formulate two queries, subsequently lever-
debates to validate individual claims, ensuring aging these queries to retrieve evidence with Google
meticulous verification outcomes. Experimen- API or the provided knowledge. Multi-agent Verifica-
tal results across three generative tasks demon- tion, we propose the Markov Chain-based multi-agent
strate that our approach achieves significant debate verification framework, which can emulate hu-
improvements over baselines. man behavior to enhance model outputs and improve
reasoning capabilities. Our main contributions focus on
1 Introduction the process of Multi-agent Verification.
The continuous evolution of large language models
(LLMs) has significantly expanded language pro-
cessing capabilities across diverse domains (Wei during the training process (Lee et al., 2022; Tou-
et al., 2022). However, this progress introduces vron et al., 2023; Elaraby et al., 2023; Wu et al.,
challenges, such as the substantial cost associated 2023b), rendering them unsuitable for large lan-
with updating model parameters and inherent defi- guage models with agnostic parameters and these
ciencies in reasoning (Ji et al., 2023; Zhang et al., methods often incur considerable costs. Conse-
2023b; Zheng et al., 2023). This has led to the quently, researchers have explored post-processing
generation of inaccurate content, known as halluci- approaches (Gao et al., 2023; Peng et al., 2023;
nation, particularly concerning potent yet opaque Chern et al., 2023; Vu et al., 2023; Gero et al.,
models like ChatGPT and GPT-4 (OpenAI, 2023). 2023) involving hallucination detection or correc-
Hallucination detection has become a focal point tion post-content generation. Notably, these meth-
in addressing these challenges. Existing methods ods typically focus on problem decomposition and
often necessitate costly and intricate interventions evidence retrieval, emphasizing simple prompting
∗
Equal contribution. during individual verification. We posit that the ver-
†
Corresponding authors: Dongyan Zhao and Rui Yan. ification accuracy is pivotal compared to problem
decomposition in LLMs. (Kryscinski et al., 2020; Maynez et al., 2020; Goyal
To address these challenges, we present a fact- and Durrett, 2021), dialogue (Das et al., 2022),
checking process to enhance the accuracy of hal- question-answering (Longpre et al., 2021), and
lucination detection. As shown in Figure 1, which machine translation (Xu et al., 2023a). These ap-
involves three stages: claim detection, evidence proaches primarily aimed to identify discrepancies
retrieval, and multi-agent verification. In claim between the generated content and the input, as
detection, our approach involves the extraction well as internal inconsistencies within the gener-
of claims from extensive responses by prompting ated content. However, they were often tailored
ChatGPT, decomposing the intricate problem into specifically to task-specific models, lacking gener-
smaller components. Evidence retrieval involves alizability. There were also fact-checking endeav-
generating queries based on claims for retrieval. ors that aimed to identify discrepancies between the
Subsequently, we retrieve the corresponding ev- generated content and real-world facts. This was
idence based on these generated queries. In the typically accomplished through three steps (Guo
multi-agent verification stage, we innovatively pro- et al., 2022): Claim Detection, Evidence Retrieval,
pose a Markov Chain-based multi-agent debate ver- and Verdict Prediction. With the advent of large
ification framework, which leverages the robust language models, some works (Gao et al., 2023;
capabilities of multi-agent systems to simulate hu- Li et al., 2023a) tackled the task of hallucination
man behavior. This approach involves deploying detection by prompting the large language models
diverse agents in Markov Chain debates to verify directly. In addition to task-specific approaches,
individual claims, thus providing a nuanced and there are hallucination detection methods specifi-
flexible validation process. Following the verifica- cally designed for LLMs. For example, some meth-
tion of each claim using our method, the collective ods assess hallucination detection by examining the
judgment of all claims contributes to the detection consistency of sampled examples (Manakul et al.,
of hallucinations in the original response. 2023; Zhang et al., 2023a). Our work is fundamen-
We conduct extensive experiments across three tally based on the fact-check framework. We trans-
generative tasks, including question-answering, fer the Verdict Prediction stage to the Multi-agent
Summarization, and Dialogue, demonstrating the Verification to improve the precision of validation.
effectiveness of our approach. Verification out-
comes are meticulously analyzed and compared 2.2 Hallucination Mitigation
against existing methods to ascertain the superior- LLMs have demonstrated significant potential re-
ity of our approach. In summary, our contributions cently. However, they have not been able to com-
can be summarized as follows: pletely eliminate the occurrence of hallucinations
(Zheng et al., 2023). The extended text produced
• We propose a versatile hallucination detec- by these large models, which encompasses more di-
tion process applicable to multiple generation verse content and often introduces external knowl-
tasks for improving verification accuracy. edge, renders traditional methods for hallucination
• We introduce a Markov Chain-based multi- mitigation less effective. Consequently, a plethora
agent debate verification framework that sim- of works dedicated to addressing hallucination miti-
ulates human discussion. gation in LLMs have emerged. Various approaches
are presented to mitigate hallucinations at different
• Experiments conducted on three generative stages of their application within the LLM life cy-
tasks show that our proposed framework out- cle (Zhang et al., 2023b), including the pre-training
performs baselines. phase of large models (Lee et al., 2022; Touvron
et al., 2023), the SFT phase (Chen et al., 2023;
2 Related Work Elaraby et al., 2023), the alignment phase (Wu
et al., 2023b; Casper et al., 2023), and the decoding
2.1 Hallucination Detection phase (Li et al., 2023b; Chuang et al., 2023). Im-
Before the emergence of large language mod- plementing these methods necessitates adjustments
els, hallucination detection was a significant topic to the model’s parameters and requires a certain
within the field of natural language processing. Pre- amount of training data, incurring some overhead.
vious efforts primarily focused on detecting hallu- Numerous endeavors have been undertaken to miti-
cinations in various tasks such as summarization gate hallucinations in content generated by black-
 Preparation 1
Answer=False

Debate roles

Debate Mode1
Answer
=True
Trust Skeptic Leader Answer=False

Agent Customization
Markov Chain- Initial
Preparation 2 based Debates
answer
Debate Mode1：Trust-initiated

Answer=True
Answer
=False
Debate Mode2
Trust Skeptic Leader
Debate Mode2: Skeptic-initiated

Answer=True
Skeptic Trust Leader

Debate Modes Customization Multi-agent Chain of Verification

Figure 2: Overview of the proposed multi-agent debate verification framework for hallucination detection. There
are two preparations before multi-agent debate verification. In preparation 1 (Agent Customization), we define
three different debate agent roles, including Trust, Skeptic, and Leader roles. In preparation 2 (Debate Modes
Customization), we assume that the debate process consists of two modes: Trust agent-initiated discussion (Trust-
Skeptic-Leader) and Skeptic agent-initiated discussion (Skeptic-Trust-Leader). Then, in Multi-agent Chain of
Verification, our validation process can be regarded as a Markov chain, which continually oscillates between these
two debate modes to arrive at an optimal judgment.

box models, such as leveraging external knowledge tween agents, which may pose concerns regarding
bases or tools (Gao et al., 2023; Peng et al., 2023; the self-correction approach (Huang et al., 2023).
Chern et al., 2023; Vu et al., 2023) and adopting Therefore, the objective of our work is to facilitate
self-refining approaches (Gero et al., 2023). Our ap- flexible discussions among multiple agents based
proach also centers on hallucination mitigation for on existing facts, aiming to detect and mitigate hal-
black-box models, introducing a distinctive multi- lucinations in the generated content of language
agent method to augment its effectiveness. models.

2.3 Multi-agent in LLMs

3 Method
In recent years, there has been a significant in-
crease in the size of models and the amount of The primary objective of our study is to detect hal-
training data used, resulting in the exceptional per- lucinations in the content generated by the model.
formance of large language models (LLMs) across To accomplish this, we adhere to the conventional
various tasks. As a result, researchers have ex- fact-checking process and make some modifica-
plored the use of LLMs as agents to simulate hu- tions. The process is structured into three distinct
man behavior, leading to the development of in- stages: Claim Detection, Evidence Retrieval, and
fluential projects such as Generative Agents (Park Multi-agent Verification. This systematic approach
et al., 2023), Ghost in the Minecraft (Zhu et al., enables the dissection of a complex problem into
2023), GPT-Bargaining (Fu et al., 2023) and Were- more manageable components. It has come to our
wolf game (Xu et al., 2023b). There are also some attention that in certain fact-checking procedures,
efforts involve multiple agents engaging in debates despite the accurate extraction of claims and the
to improve the reasoning capabilities (Liang et al., acquisition of robust evidence, verification errors
2023; Du et al., 2023; Xiong et al., 2023) or address persist in the final stage, undermining the efficacy
issues related to hallucinations (Du et al., 2023; Co- of preceding efforts.
hen et al., 2023). However, existing methods for Therefore, we propose a novel multi-agent de-
hallucination detection and mitigation of LLMs bate verification framework for hallucination detec-
solely rely on natural language interactions be- tion, the overview of which is shown in Figure 2.
An anthropomorphic debate process based on the Du et al., 2023), two critical aspects remain under-
Markov chain is designed to be applicable across explored within the realm of hallucination detec-
various generative tasks in the proposed method, tion.
bolstering the accuracy of verification. Subsequent
sections will expound on each of these three stages 1) Application to Verification: Few studies
individually, with a particular emphasis on our in- have directly applied the multi-agent approach
novative approach in the third stage. to the task of verification, they more concen-
trate on the decomposition of the complicated
3.1 Claim Detection samples. Recognizing this research gap, our
In the stage of claim detection, we employ the work aims to bridge it by introducing the
methodology utilized in Factool (Chern et al., multi-agent debate verification framework.
2023), leveraging large language models such
2) Flexible Debate Process: Existing methods
as ChatGPT. Harnessing the robust instruction-
in debate often adhere to a fixed process, un-
following capabilities of LLMs empowers us to ad-
like human debates where participants dynam-
dress the challenge of dissecting intricate responses.
ically adjust their arguments based on prior
Nevertheless, detecting the hallucinations in state-
outcomes. Our proposed approach takes in-
ments lacking adequate information is futile and
spiration from the Markov chain, where the
could impede overall judgment. Moreover, specific
selection of the current state depends on the re-
tasks may demand the concatenation of the model’s
sults of a limited set of preceding states. This
responses with particular input information to for-
debate mode is more similar to the discussion
mulate an informative claim, necessitating supple-
between humans.
mentary processing. Detailed explanations of these
processing methods are provided in the experimen- In summary, our multi-agent debate verifica-
tal implementation section §4.1.2. tion framework ingeniously adapts the multi-agent
paradigm to the hallucination detection task. By in-
3.2 Evidence Retrieval
fusing flexibility into the debate process and draw-
Upon extraction of claims, a retrieval methodology ing inspiration from the Markov chain, our goal
is employed to ascertain corresponding evidence. is to enhance the accuracy and adaptability of the
Drawing inspiration from Factool’s (Chern et al., verification process when assessing the veracity of
2023) strategy in Knowledge Base Question An- claims based on evidence.
swering (KBQA) tasks, we prompt ChatGPT to for- The key point in our method lies in the definition
mulate two queries, subsequently leveraging these of states and the transition mechanisms.
queries to retrieve evidence. In instances where per-
tinent knowledge is absent, we employ the Google 3.3.1 States
API to retrieve data from the internet. Conversely, Agents To comprehend the definition of states,
when dealing with data accompanied by provided it is imperative to elucidate the roles assumed by
knowledge, we either consider the length of the the diverse agents under consideration. We en-
knowledge as direct evidence or encode it for local gage three distinct agents: Trust, Skeptic, and
retrieval. Leader. These agents collectively share the com-
monality of assimilating perspectives from one or
3.3 Multi-agent Verification more antecedent agents. They meticulously scru-
We propose a Markov Chain-based multi-agent de- tinize these perspectives, grounded in claims and
bate verification framework. Our investigations re- evidence amassed in preceding sections, express
veal significant potential in employing multi-agent concurrence or dissent, and proffer their own view-
systems to emulate human behavior (Park et al., points accompanied by factual assessments of the
2023; Zhu et al., 2023), particularly in the do- claims. The differentiation among these agents lies
main of fact-checking claims grounded in evidence. in their inclination toward antecedent viewpoints.
The effectiveness of addressing this task is notably The Trust agent predominantly leans towards ac-
heightened through the use of multi-agent debates. cepting the perspectives of the preceding agent,
Despite considerable advancements in leveraging thereby bolstering their credibility. Conversely, the
multi-agent debates to enhance model outputs and Skeptic agent challenges the perspectives of the
improve reasoning capabilities (Liang et al., 2023; antecedent agent, diligently seeking to pinpoint
inconsistencies between viewpoints and support- human intuition, acknowledging the potential for
ing evidence. The Leader agent amalgamates the diverse perspectives in debating a given matter.
perspectives of two agents, critically examines the Our transition probabilities are as follows:
rational and irrational facets, and ultimately formu-
lates its own viewpoint. We implement agents with P r (S2 |R = T rue) = 1 (1)
different personas through various prompts. Details
can be found in Appendix A.1. The configuration P r (S1 |R = F alse) = 1 (2)
of these agents, arranged in various sequences, con-
stitutes the states delineated in our approach. R represents the judgment obtained from the
previous state. Specifically, our chosen transition
States We need to precisely define the states men- method operates as follows: if the preceding state
tioned earlier. According to the definition of the deems the current claim as factual, we transition
Markov chain, we require an initial state to initiate to S2 . Our objective is to engage in a rigorous
our verification chain. Each agent must analyze the discussion, analyzing and questioning the claim
perspectives of preceding agents, necessitating an only in the absence of contradictions in the pre-
initial agent to furnish the primary answer for sub- vious state. The goal is to identify and address
sequent debate. This initial state is characterized by potential loopholes. If none are found, the Trust
the initial agent, labeled as S0 , and our verification agent can reasonably conclude acceptance of the
chain unfolds from this state. answer, leading to the convergence of the entire
We predominantly have two ordinary states, each chain. Conversely, when the preceding state cate-
comprising three agents. These states can be re- gorizes the claim as non-factual, we transition to
garded as two distinct discussion modes. The first S1 . In essence, we initially reinforce the credibility
is the Trust agent-initiated discussion, labeled as of this judgment, confirming the validity of skepti-
S1 , following the sequence Trust-Skeptic-Leader. cism. By enhancing the credibility of this opinion,
This mode aims to bolster the credibility of the if subsequent skepticism from the Skeptic agent is
preceding viewpoint before introducing skepticism. challenging, we can reasonably conclude the accu-
The second state is initiated by the Skeptic agent, racy of this judgment, leading to the convergence
designated as S2 , with the sequence Skeptic-Trust- of the chain.
Leader. This mode leans towards questioning the Therefore, our overall process unfolds in the
credibility of the previous viewpoints before further following manner: Initially, an initial answer is
analyzing the skeptical perspective. Our verifica- obtained from the initial state S0 . Based on this
tion chain continually oscillates between these two answer, the first transition to either S1 or S2 is
debate modes to arrive at an optimal judgment. made. Subsequent transitions rely solely on the
To prevent the chain from infinitely extending, judgment of the preceding state, continuing until a
a termination state is essential. Analogous to hu- consensus is reached among the three agents within
man debates concluding when opinions align, our a state, culminating in the final verification result.
termination condition is similar. If, within a state,
the three agents reach a consensus, the chain ter- 4 Experiments
minates. When the Skeptic agent fails to identify
points of contention, and the Leader, after scrutiniz- We conducted experiments encompassing three
ing their opinions, has no objections, yielding the generative tasks: Knowledge-Based Question An-
same judgment, we consider the debate concluded. swering (KB-QA), Dialogue, and Summarization.
Additionally, we have imposed a maximum limit
on verification rounds to constrain the length of the 4.1 Experimental Setup
chain. For all three tasks, we prompt the ChatGPT to
execute claim extraction, query generation, and
3.3.2 Transition multi-agent debate verification. The verification
Transitioning between states is a critical aspect of process is iterated a minimum of 2 rounds, and
our methodology, following the definition of states. 10 snippets of evidence are extracted. The chosen
The primary criterion guiding these transitions in transition method involved switching to the skep-
our approach is the verification result of a claim by tic agent when the response was determined to be
the preceding state. This methodology aligns with True.
 Datasets Positive Negative HaluEval (Li et al., 2023a), where answers some-
Factool QA 23 27
HaluEval QA 75 75 times are the single entity, such as "What American
HaluEval Summarization 25 25 quartery lifestyle magazine did Hearst Shkelev Me-
HaluEval Dialogue 80 70 dia also publish? Departures.", we concatenated
Table 1: The Number of positive and negative samples
answers and questions to form QA pairs. Subse-
in different datasets. quently, we directly applied the multi-agent debate
verification to these QA pairs, utilizing the pro-
vided knowledge in the dataset as evidence.
4.1.1 Datasets and Baselines
In this paper, we perform experiments on three Summarization The model-generated summary
different tasks, including Question-Answer (QA), was treated as a response, decomposed into multi-
Summarization, and Dialogue. The experimental ple claims, and each claim was verified individu-
datasets are derived from the following two canoni- ally. The corresponding document to the summary
cal databases: served as evidence. To mitigate excessively long
input queries, each sentence of the document was
• Factool (Chern et al., 2023): The Factprompts encoded separately, along with the query. The top
data comprises real-world questions with re- 10 most similar sentences were selected as evidence
sponses generated by ChatGPT, along with for the current claim.
Factool-annotated claims extracted from these
responses. Dialogue In the course of the dialogue task, we
encountered challenges associated with the extrac-
• HaluEval (Li et al., 2023a): HaluEval con- tion of claims. Dialogue responses frequently in-
stitutes a substantial collection of sampling- corporated substantial subjective viewpoints such
then-filtering generated and human-annotated as "The last time that they made it to Super Bowl
hallucinated samples, serving as an evalua- was in 2005. Are you a basketball fanatic too?",
tion metric for language model performance rendering the fact-checking of the factual accu-
in recognizing hallucination. racy of such subjective statements less meaningful.
We randomly selected 150, 50, and 150 samples To mitigate this challenge, we introduced a pre-
from the three tasks of HaluEval for testing pur- processing step wherein we directed ChatGPT to
poses. The selection of samples was contingent eliminate subjective portions from its responses
upon the complexity of task responses, with sum- prior to claim extraction, so the previous sentence
marization outputs being more intricate. Owing to becomes: "The last time that they made it to the Su-
the necessity of decomposing summarization into per Bowl was in 2005.". This approach allowed us
a greater number of claims, the extracted quantity to retain only the informative segments for subse-
is comparatively smaller than that of the other two quent verification. Additionally, in the verification
tasks. The positive and negative instances within process during claim extraction, we employed the
the dataset were randomly sampled using a binary dialogue history and external knowledge as sup-
distribution with a probability of 0.5. The resulting porting evidence.
data distribution is presented in Table 1. 4.2 Performance Analysis
We compared the Factool method, the few-shot
prompting method in HaluEval, the self-check The experimental results are presented in Table 2
method (Chern et al., 2023), and our approach. and Table 3. Table 2 shows the performance of
our method on Factool (Chern et al., 2023), pre-
4.1.2 Implementataion Details senting results at both the claim and response lev-
KB-QA For intricate and information-rich QA els. According to Table 2, we can observe that our
data, such as that in Factool (Chern et al., 2023), we proposed method can consistently achieve optimal
decomposed answers into multiple atomic claims accuracy when compared to various approaches.
and conducted multi-agent debate verification on Table 3 displays the test results on the HaluE-
each claim. If one of the claims is hallucinated, the val (Li et al., 2023a) dataset, from which we can
origin answer is judged to be non-factual. As Fac- observe that: Our method demonstrates optimal
tool data lacked corresponding evidence, Google accuracy, excelling in most metrics in all three
search was employed to retrieve evidence for verifi- tasks, Notably, in the three tasks of this dataset, our
cation. In the case of simpler QA data, as found in method exhibits a relatively low recall score. This
 Claim-Level Response-Level
Method
Acc. R P F1 Acc. R P F1
Self-Check (0) 75.54 90.40 80.00 84.88 54.00 60.87 50.00 54.90
Self-Check (3) 69.53 81.36 79.12 80.23 54.00 47.83 50.00 48.89
FACTOOL 74.25 73.45 90.91 81.25 64.00 43.48 66.67 52.63
Our Method 77.68 80.79 88.82 84.62 72.00 52.17 80.00 63.15

Table 2: Accuracy(%), Recall(%), Precision(%), F1(%) of four methods on Dataset Factool (Chern et al., 2023)
used. Claim-Level denotes the results evaluated on all annotated claims, and Response-Level denotes the results
evaluated on the origin responses. The best scores are highlighted in bold.

QA Summarization Dialogue
Method
Acc. R P F1 Acc. R P F1 Acc. R P F1
HaluEval 56.00 77.33 54.21 63.74 58.00 100.0 54.35 70.42 68.00 75.71 63.10 68.83
FACTOOL 67.33 86.67 62.50 72.63 64.00 48.00 70.59 57.14 74.67 70.00 74,24 72.06
Ours 70.67 82.67 66.67 73.81 70.00 64.00 72.73 68.09 76.00 62.86 81.48 70.97

Table 3: The results for our method and baseline on HaluEval (Li et al., 2023a) Dataset. We conducted experiments
on three tasks: QA, Summarization, and Dialogue. The best scores are highlighted in bold.

can be attributed to our approach, which involves

Method Acc. R P F1
questioning claims verified as factual, thereby en-
Always Skeptic 65.00 84.21 59.26 69.57
suring the precise detection of errors when claims Always Trust 68.75 84.21 62.75 71.91
are misclassified. However, this approach also re- T rue → Trust 67.50 89.47 60.71 72.34
sults in misjudging some claims that inherently lack T rue → Skeptic 70.00 86.84 63.46 73.33
hallucinations as non-factual. This phenomenon is
Table 4: Comparison of Different Transition Meth-
further elucidated in § 4.3.
ods. We evaluate the influence of transition methods on
80 QA samples, setting the minimum debate rounds to
4.3 Ablation Study 2. The best scores are highlighted in bold.

Transition Methods We assessed the impact of

distinct transition methods. From the QA section Minimum Rounds of Debate We explored the
of HaluEval (Li et al., 2023a), we extract 80 sam- influence of different numbers of minimum debate
ples to evaluate the impact of four transition meth- rounds on the outcomes. We examined three dis-
ods: transitioning to S2 when the preceding state tinct tasks using the previously extracted HaluEval
deemed the current claim devoid of hallucination data (Li et al., 2023a), varying the number of min-
(T rue → Skeptic), transitioning to S1 when the imum debate rounds from 0 to 3. Employing the
preceding state deemed the current claim devoid of T rue → Skeptic transition method, the results,
hallucination (T rue → Trust), consistently tran- illustrated in Figure 3, generally exhibit enhanced
sitioning to S1 or S2 irrespective of the preceding performance when the number of minimum rounds
state’s judgment about the claim. The results, pre- is set to 1 or 2, with a discernible decrease in effi-
sented in Table 4, reveal that T rue → Skeptic cacy when the number of minimum rounds is set
achieved optimal performance across three met- to 3.
rics. This is primarily attributed to the fact that this
transition method endeavors to challenge claims Comparison with Non-GPT Method In the
deemed factual in the preceding state, subsequently multi-agent verification stage of the experiment in
scrutinizing for potential oversights. In accor- the Factool dataset, we employed the WeCheck(Wu
dance with the details presented in § 4.2, this phe- et al., 2023a) method to conduct an ablation study,
nomenon results in a lower recall score than the showcasing the benefits of our approach. We held
T rue → Trust method, concurrently demonstrat- the initial two steps constant, utilizing the Factool
ing an elevated precision value. method to extract claims and retrieve evidence.
 glish Mastiff having a wider range." In Table 10,
Accuracy V.S. Debate Rounds
0.77 the three agents engage in discussions to highlight
0.75 the insufficient evidence supporting "The Landseer
has a limited range of colors." However, they fail to
0.73
infer the contradiction with the evidence suggesting
Detection accuracy

0.71 "the English Mastiff has a wider range." In Table

0.69 11, the agent, post-discussion, identifies both of
these deficiencies. These observations indicate that
0.67
larger language models, owing to their enhanced
0.65 KBQA
Summarization reasoning capabilities, yield better results when em-
Dialogue
0.63 ploying our method. Furthermore, it highlights that
0 1 2 3
Debate Minimum Rounds in some cases, a single round of debate may not
reveal all inconsistencies between claim and evi-
Figure 3: Comparison of Different Minimum Debate dence, emphasizing why sometimes increasing the
Rounds. We evaluate the influence of the minimum de- minimum debate rounds can improve effectiveness.
bate rounds on samples we utilized in §4.1.1, setting the
transition method to T rue → Skeptic. The x-axis rep-
5 Conclusion
resents different minimum debate rounds, whereas the
y-axis signifies the corresponding detection accuracy. In this paper, our purpose is to improve the accu-
racy of hallucination detection in content generated
Method Acc. R P F1 by large language models. Simultaneously, we as-
Wecheck 65.23 64.41 86.36 73.78
pired to extend this enhancement beyond particular
Our method 77.68 80.79 88.82 84.62 generative tasks. To fulfill these objectives, we in-
troduce a versatile framework for hallucination de-
Table 5: Comparison with Non-GPT Method. We tection and propose the Markov Chain-based multi-
compare our method with Non-GPT Method Wecheck agent debate verification framework. Our proposed
on the Factool dataset. The best scores are highlighted approach demonstrates its effectiveness through
in bold.
evaluations conducted on both the Knowledge Base
Question Answering (KBQA) dataset and the ran-
Employing the claim as the hypothesis and the domly sampled HaluEval dataset. We posit that
evidence as the premise, instances with WeCheck our method demonstrates a level of generalizabil-
scores greater than or equal to 0.5 were deemed fac- ity, enabling its adaptation to other post-processing
tual. From the experimental results in Table 5, we hallucination detection or mitigation approaches
observed that compared to the non-GPT method, for better performance.
our approach exhibits significant advantages during
the verification stage. Limitations and Potential Risks
Our methodology necessitates frequent interactions
4.4 Case Study
with the API of large language models (LLMs), re-
To demonstrate the effectiveness of our approach, sulting in significant overhead. This high frequency
Table 10 and Table 11 show examples of the hallu- of API calls increases the cost and reduces response
cination detection process for a Question-Answer speed, which may limit its practicality in real-world
(QA) sample. In Table 10, the debating agent is scenarios. Nevertheless, this approach provides an
based on the GPT-3.5-turbo model, whereas Table accessible option for users lacking the infrastruc-
11 utilizes GPT-4 as the base model. ture to implement large open-source models.
When the debate starts, the initial agent gener- Furthermore, the distinctiveness among prompts
ates an initial opinion based on the QA pair and for different agents primarily centers on role def-
the corresponding evidence. If no debate ensues, inition, while other aspects display considerable
the initial opinion solidifies as the final answer. similarity. This occasionally leads to the partial
However, this approach overlooks both the insuffi- repetition of opinions from the preceding agent. As
ciency of evidence to support the claim that "The exemplified by the two instances in Appendix A.2,
Landseer has a limited range of colors" and the this phenomenon could be substantially alleviated
contradiction with evidence concerning "the En- by enhancing the performance of the base model.
References Long Papers), pages 16477–16508, Toronto, Canada.
Association for Computational Linguistics.
Stephen Casper, Xander Davies, Claudia Shi,
Thomas Krendl Gilbert, Jérémy Scheurer, Javier
Zelalem Gero, Chandan Singh, Hao Cheng, Tristan Nau-
Rando, Rachel Freedman, Tomasz Korbak, David
mann, Michel Galley, Jianfeng Gao, and Hoifung
Lindner, Pedro Freire, Tony Wang, Samuel Marks,
Poon. 2023. Self-verification improves few-shot clin-
Charbel-Raphaël Segerie, Micah Carroll, Andi Peng,
ical information extraction. In ICML 3rd Workshop
Phillip Christoffersen, Mehul Damani, Stewart
on Interpretable Machine Learning in Healthcare
Slocum, Usman Anwar, Anand Siththaranjan, Max
(IMLH).
Nadeau, Eric J. Michaud, Jacob Pfau, Dmitrii
Krasheninnikov, Xin Chen, Lauro Langosco, Peter
Hase, Erdem Bıyık, Anca Dragan, David Krueger, Tanya Goyal and Greg Durrett. 2021. Annotating and
Dorsa Sadigh, and Dylan Hadfield-Menell. 2023. modeling fine-grained factuality in summarization.
Open problems and fundamental limitations of In Proceedings of the 2021 Conference of the North
reinforcement learning from human feedback. American Chapter of the Association for Computa-
tional Linguistics: Human Language Technologies,
Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa pages 1449–1462, Online. Association for Computa-
Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srini- tional Linguistics.
vasan, Tianyi Zhou, Heng Huang, and Hongxia Jin.
2023. Alpagasus: Training a better alpaca with fewer Zhijiang Guo, Michael Schlichtkrull, and Andreas Vla-
data. chos. 2022. A survey on automated fact-checking.
Transactions of the Association for Computational
I-Chun Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Linguistics, 10:178–206.
Kehua Feng, Chunting Zhou, Junxian He, Graham
Neubig, and Pengfei Liu. 2023. Factool: Factuality Jie Huang, Xinyun Chen, Swaroop Mishra,
detection in generative ai – a tool augmented frame- Huaixiu Steven Zheng, Adams Wei Yu, Xiny-
work for multi-task and multi-domain scenarios. ing Song, and Denny Zhou. 2023. Large language
models cannot self-correct reasoning yet.
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon
Kim, James Glass, and Pengcheng He. 2023. Dola: Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan
Decoding by contrasting layers improves factuality Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea
in large language models. Madotto, and Pascale Fung. 2023. Survey of halluci-
Roi Cohen, May Hamri, Mor Geva, and Amir Glober- nation in natural language generation. ACM Comput.
son. 2023. Lm vs lm: Detecting factual errors via Surv., 55(12).
cross examination.
Wojciech Kryscinski, Bryan McCann, Caiming Xiong,
Souvik Das, Sougata Saha, and Rohini Srihari. 2022. and Richard Socher. 2020. Evaluating the factual
Diving deep into modes of fact hallucinations in dia- consistency of abstractive text summarization. In
logue systems. In Findings of the Association for Proceedings of the 2020 Conference on Empirical
Computational Linguistics: EMNLP 2022, pages Methods in Natural Language Processing (EMNLP),
684–699, Abu Dhabi, United Arab Emirates. Associ- pages 9332–9346, Online. Association for Computa-
ation for Computational Linguistics. tional Linguistics.

Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pas-
Tenenbaum, and Igor Mordatch. 2023. Improving cale N Fung, Mohammad Shoeybi, and Bryan Catan-
factuality and reasoning in language models through zaro. 2022. Factuality enhanced language models for
multiagent debate. open-ended text generation. In Advances in Neural
Information Processing Systems, volume 35, pages
Mohamed Elaraby, Mengyin Lu, Jacob Dunn, Xueying 34586–34599. Curran Associates, Inc.
Zhang, Yu Wang, Shizhu Liu, Pingchuan Tian, Yup-
ing Wang, and Yuxuan Wang. 2023. Halo: Estima- Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun
tion and reduction of hallucinations in open-source Nie, and Ji-Rong Wen. 2023a. Halueval: A large-
weak large language models. scale hallucination evaluation benchmark for large
Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. language models.
2023. Improving language model negotiation with
self-play and in-context learning from ai feedback. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter
Pfister, and Martin Wattenberg. 2023b. Inference-
Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony time intervention: Eliciting truthful answers from a
Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent language model.
Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and
Kelvin Guu. 2023. RARR: Researching and revising Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang,
what language models say, using language models. Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and
In Proceedings of the 61st Annual Meeting of the Shuming Shi. 2023. Encouraging divergent thinking
Association for Computational Linguistics (Volume 1: in large language models through multi-agent debate.
Shayne Longpre, Kartik Perisetla, Anthony Chen, Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel,
Nikhil Ramesh, Chris DuBois, and Sameer Singh. Barret Zoph, Sebastian Borgeaud, Dani Yogatama,
2021. Entity-based knowledge conflicts in question Maarten Bosma, Denny Zhou, Donald Metzler, Ed H.
answering. In Proceedings of the 2021 Conference Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy
on Empirical Methods in Natural Language Process- Liang, Jeff Dean, and William Fedus. 2022. Emer-
ing, pages 7052–7063, Online and Punta Cana, Do- gent abilities of large language models. Transactions
minican Republic. Association for Computational on Machine Learning Research. Survey Certifica-
Linguistics. tion.

Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Sujian
2023. Selfcheckgpt: Zero-resource black-box hal- Li, and Yajuan Lyu. 2023a. WeCheck: Strong factual
lucination detection for generative large language consistency checker via weakly supervised learning.
models. In Proceedings of the 61st Annual Meeting of the
Association for Computational Linguistics (Volume
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and 1: Long Papers), pages 307–321, Toronto, Canada.
Ryan McDonald. 2020. On faithfulness and factu- Association for Computational Linguistics.
ality in abstractive summarization. In Proceedings
of the 58th Annual Meeting of the Association for Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane
Computational Linguistics, pages 1906–1919, On- Suhr, Prithviraj Ammanabrolu, Noah A. Smith, Mari
line. Association for Computational Linguistics. Ostendorf, and Hannaneh Hajishirzi. 2023b. Fine-
grained human feedback gives better rewards for lan-
OpenAI. 2023. Gpt-4 technical report. guage model training.
Kai Xiong, Xiao Ding, Yixin Cao, Ting Liu, and Bing
Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Qin. 2023. Examining inter-consistency of large lan-
Meredith Ringel Morris, Percy Liang, and Michael S. guage models collaboration: An in-depth analysis via
Bernstein. 2023. Generative agents: Interactive sim- debate.
ulacra of human behavior.
Weijia Xu, Sweta Agrawal, Eleftheria Briakou, Mari-
Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, anna J. Martindale, and Marine Carpuat. 2023a. Un-
Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou derstanding and detecting hallucinations in neural
Yu, Weizhu Chen, and Jianfeng Gao. 2023. Check machine translation via model introspection. Trans-
your facts and try again: Improving large language actions of the Association for Computational Linguis-
models with external knowledge and automated feed- tics, 11:546–564.
back.
Yuzhuang Xu, Shuo Wang, Peng Li, Fuwen Luo, Xi-
Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- aolong Wang, Weidong Liu, and Yang Liu. 2023b.
bert, Amjad Almahairi, Yasmine Babaei, Nikolay Exploring large language models for communication
Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti games: An empirical study on werewolf.
Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton
Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jiaxin Zhang, Zhuohang Li, Kamalika Das, Bradley A.
Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Malin, and Sricharan Kumar. 2023a. Sac3 : Reliable
Cynthia Gao, Vedanuj Goswami, Naman Goyal, An- hallucination detection in black-box language models
thony Hartshorn, Saghar Hosseini, Rui Hou, Hakan via semantic-aware cross-check consistency.
Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa,
Isabel Kloumann, Artem Korenev, Punit Singh Koura, Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu,
Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Di- Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang,
ana Liskovich, Yinghai Lu, Yuning Mao, Xavier Mar- Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei
tinet, Todor Mihaylov, Pushkar Mishra, Igor Moly- Bi, Freda Shi, and Shuming Shi. 2023b. Siren’s song
bog, Yixin Nie, Andrew Poulton, Jeremy Reizen- in the ai ocean: A survey on hallucination in large
stein, Rashi Rungta, Kalyan Saladi, Alan Schelten, language models.
Ruan Silva, Eric Michael Smith, Ranjan Subrama-
Shen Zheng, Jie Huang, and Kevin Chen-Chuan Chang.
nian, Xiaoqing Ellen Tan, Binh Tang, Ross Tay-
2023. Why does chatgpt fall short in providing truth-
lor, Adina Williams, Jian Xiang Kuan, Puxin Xu,
ful answers?
Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan,
Melanie Kambadur, Sharan Narang, Aurelien Ro- Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Wei-
driguez, Robert Stojnic, Sergey Edunov, and Thomas jie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu,
Scialom. 2023. Llama 2: Open foundation and fine- Xiaogang Wang, Yu Qiao, Zhaoxiang Zhang, and
tuned chat models. Jifeng Dai. 2023. Ghost in the minecraft: Gener-
ally capable agents for open-world environments via
Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry large language models with text-based knowledge
Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny and memory.
Zhou, Quoc Le, and Thang Luong. 2023. Freshllms:
Refreshing large language models with search engine
augmentation.
A Appendix
A.1 Prompts
Table 6, 7, 8 and 9 enumerate various prompts
employed in our experimental design, including
prompts for establishing different roles for the
agent and prompts for eliminating subjective opin-
ions from dialogue responses.

A.2 Debate examples

In Table 10 and Table 11, we present two instances
of Multi-Agent Debate Verification on a HaluE-
val QA sample. The agents involved in these two
instances employ distinct base models: GPT-3.5-
turbo and GPT-4. In Table 12, we provide a com-
prehensive breakdown of our verification method’s
inference process for better understanding, detail-
ing the inputs and outputs of each agent.
You are the *Trust* agent of the three agents. Your task is to trust the previous agent’s opinions as much
as possible and to further expand upon it.
You are given the opinions generated by a previous agent. Refer to the claim[text] and evidence[evidences]
to analyse the opinions[previous opinions] of the previous agent. Carefully examine whether the corre-
sponding evidences support the statements put forward by the previous agent. If you believe any part of
them is accurate, please further analyze it based on that.
Then assess the factuality of the initial claim [text] based on the provided information [evidences] and the
previous agent’s opinions.
DO NOT REPEAT the previous agent’s opinion, you should develop your own perspective based on theirs.
Take the previous agent’s opinions as a reference rather than copying it directly.
The response should be a dictionary with three keys - "opinion", "factuality", "Error severity" which
correspond to the your opinion, whether the given text is factual or not (Boolean - True or False), the
severity level of claim error(Integer - Range from 0 to 5).
The definitions of different levels of error severity are as follows:

0. No Error (Grade 0): This is when the claim is entirely factual and accurate, with no errors or
inaccuracies.

1. Minor Errors (Grade 1): These are small and insignificant errors that do not significantly alter
the essence or validity of the claim. For example, minor spelling mistakes, incorrect dates or small
numerical discrepancies.

2. Moderate Errors (Grade 2): These are errors that have some impact on the validity of the claim but
do not reverse its overall meaning. For example, wrong terminologies, improper use of statistics or
minor deviations from the evidence.

3. Major Errors (Grade 3): These are errors that have a significant impact on the validity of the claim.
These errors may lead to significant misunderstandings or misinterpretations. Examples could be
gross exaggeration or underestimation, misuse of expert authority, or manipulation of context.

4. Critical Errors (Grade 4): These are errors that completely negate or invalidate the claim. The
evidence contradicts the claim so fundamentally that its veracity is entirely undermined. For example,
attribution of a quote or event to the wrong person or an incorrect recap of a major event.

5. False Claim (Grade 5): This is when claims are entirely made up or intentionally deceptive, without
any grounding in evidence. They are blatant lies designed to mislead or deceive, fabricating events,
people, or statements that do not exist or never occurred.

YOU SHOULD ONLY RESPONSE IN FORMAT AS DESCRIBED BELOW. DO NOT RETURN

ANYTHING ELSE. START YOUR RESPONSE WITH ’{{’.
[response format]:
{{
"opinion": "First analyze the opinions of the previous agent, point out what you think is correct or incorrect
in its opinions and explain the reasons. Remember that you should trust previous agent’s opinion as much
as possible. And then describe your opinion about factuality of the claim[text] based on the evidences
[evidences]. Your opinions should be supported by corresponding evidences. DO NOT REITERATE
THE PREVIOUS AGENT’S OPINIONS [previous opinions].",
"factuality": True if the given text is factual, False otherwise.,
"Error severity": Integer - Range from 0 to 5. The severity level of claim error. According to the definitions
of levels of the error severity, please carefully compare "claim" and "evidence" and provide appropriate
degree.
}}

Table 6: Prompt For Trust Agent

You are the *Skeptic* agent of the three agents. "Skeptic" means you must question the previous agent’s
opinions by scrutinizing the available data [text] and [evidences] and identifying any possible errors or
misleading factors in previous agent’s opinions.
You are given the opinions generated by a previous agent. Refer to the claim[text] and evidence[evidences]
to analyse the opinions[previous opinions] of the previous agent. Carefully examine whether the corre-
sponding evidences support the statements put forward by the previous agent. If you believe any part of
its opinions is incorrect, please point it out and explain your perspective.
Then critically examine the validity of the claim [text], considering any potential biases or inconsistencies
between the information [evidences] and claim [text].
DO NOT REPEAT the previous agent’s opinion, you should develop your own perspective based on theirs.
Take the previous agent’s opinions as a reference rather than copying it directly.
The response should be a dictionary with three keys - "opinion", "factuality", "Error severity" which
correspond to the your opinion, whether the given text is factual or not (Boolean - True or False), the
severity level of claim error(Integer - Range from 0 to 5).
The definitions of different levels of error severity are as follows:

0. No Error (Grade 0): This is when the claim is entirely factual and accurate, with no errors or
inaccuracies.

1. Minor Errors (Grade 1): These are small and insignificant errors that do not significantly alter
the essence or validity of the claim. For example, minor spelling mistakes, incorrect dates or small
numerical discrepancies.

2. Moderate Errors (Grade 2): These are errors that have some impact on the validity of the claim but
do not reverse its overall meaning. For example, wrong terminologies, improper use of statistics or
minor deviations from the evidence.

3. Major Errors (Grade 3): These are errors that have a significant impact on the validity of the claim.
These errors may lead to significant misunderstandings or misinterpretations. Examples could be
gross exaggeration or underestimation, misuse of expert authority, or manipulation of context.

4. Critical Errors (Grade 4): These are errors that completely negate or invalidate the claim. The
evidence contradicts the claim so fundamentally that its veracity is entirely undermined. For example,
attribution of a quote or event to the wrong person or an incorrect recap of a major event.

5. False Claim (Grade 5): This is when claims are entirely made up or intentionally deceptive, without
any grounding in evidence. They are blatant lies designed to mislead or deceive, fabricating events,
people, or statements that do not exist or never occurred.

YOU SHOULD ONLY RESPONSE IN FORMAT AS DESCRIBED BELOW. DO NOT RETURN

ANYTHING ELSE. START YOUR RESPONSE WITH ’{{’.
[response format]:
{{
"opinion": "First analyze the opinions of the previous agent, point out what you think is correct or incorrect
in its opinions and explain the reasons. Remember that you should doubt the opinions of the previous
agent as much as possible. And then describe your opinion about factuality of the claim[text] based on
the evidences [evidences]. Your opinions should be supported by corresponding evidences. DO NOT
REITERATE THE PREVIOUS AGENT’S OPINIONS [previous opinions].",
"factuality": True if the given text is factual, False otherwise.,
"Error severity": Integer - Range from 0 to 5. The severity level of claim error. According to the definitions
of levels of the error severity, please carefully compare "claim" and "evidence" and provide appropriate
degree.
}}

Table 7: Prompt For Skeptic Agent

You are the *Leader* agent of the three agents. The other two agents are ’Trust’ and ’Skeptic’ agents. The
’Trust’ agent will trust the previous agent’s opinion as soon as possible and the ’Skeptic’ agent will doubt
the previous agent’s opinion as soon as possible.
You are given the opinions generated by two previous agents. Incorporate the opinions[previous opinions]
provided by the ’Trust’ and ’Skeptic’ agents to synthesize the most accurate and reliable conclusion
regarding the claim([text])’s veracity based on the [evidences]. You need to consider the characteristics of
these two agents when forming your own opinions. Assess the strengths and weaknesses of both sides,
and leverage the provided information to generate a conclusive judgment.
DO NOT REPEAT the previous agent’s opinion, you should develop your own perspective based on theirs.
Take the previous agent’s opinions as a reference rather than copying it directly.
The response should be a dictionary with three keys - "opinion", "factuality", "Error severity" which
correspond to the your opinion, whether the given text is factual or not (Boolean - True or False), the
severity level of claim error(Integer - Range from 0 to 5).
The definitions of different levels of error severity are as follows:

0. No Error (Grade 0): This is when the claim is entirely factual and accurate, with no errors or
inaccuracies.

1. Minor Errors (Grade 1): These are small and insignificant errors that do not significantly alter
the essence or validity of the claim. For example, minor spelling mistakes, incorrect dates or small
numerical discrepancies.

2. Moderate Errors (Grade 2): These are errors that have some impact on the validity of the claim but
do not reverse its overall meaning. For example, wrong terminologies, improper use of statistics or
minor deviations from the evidence.

3. Major Errors (Grade 3): These are errors that have a significant impact on the validity of the claim.
These errors may lead to significant misunderstandings or misinterpretations. Examples could be
gross exaggeration or underestimation, misuse of expert authority, or manipulation of context.

4. Critical Errors (Grade 4): These are errors that completely negate or invalidate the claim. The
evidence contradicts the claim so fundamentally that its veracity is entirely undermined. For example,
attribution of a quote or event to the wrong person or an incorrect recap of a major event.

5. False Claim (Grade 5): This is when claims are entirely made up or intentionally deceptive, without
any grounding in evidence. They are blatant lies designed to mislead or deceive, fabricating events,
people, or statements that do not exist or never occurred.

YOU SHOULD ONLY RESPONSE IN FORMAT AS DESCRIBED BELOW. DO NOT RETURN

ANYTHING ELSE. START YOUR RESPONSE WITH ’{{’.
[response format]:
{{
"opinion": "First explain your thoughts on the opinions of the two previous agents. And then describe
your opinions about the factuality of the claim[text] based on the evidences [evidences]. Your opinions
should be supported by corresponding evidences. DO NOT REITERATE ANY OF THE PREVIOUS
AGENTS’ OPINIONS [previous opinions]. With reference to the opinions of the "Trust" agent and the
"Skeptic" agent, derive a novel opinion that you believe to be correct.",
"factuality": True if the given text is factual, False otherwise.,
"Error severity": Integer - Range from 0 to 5. The severity level of claim error. According to the definitions
of levels of the error severity, please carefully compare "claim" and "evidence" and provide appropriate
degree.
}}

Table 8: Prompt For Leader Agent

You are given a piece of text, please remove sentences that you believe are totally personal opinions and
do not contain any factual information.
Your output should be the sentence after modification of the original content. If you believe the entire
sentence is a personal opinion, please output None.
Here are two examples:
[text]: My pleasure, let me know if you need more recommendations.
[response]: None
[text]: The Fault in Our Stars is a fantasy novel. Have you read it?
[response]: The Fault in Our Stars is a fantasy novel.
Now complete the following
[text]: {input}
[response]:

Table 9: Prompt for extracting effective information

Question: Does the Landseer or English Mastiff have a limited range of colors?
Answer: The Landseer has a limited range of colours, while the English Mastiff has a wider range.
Evidence: It is not to be confused with a white and black Newfoundland, which is also often called
a [Link] English Mastiff is a breed of extremely large dog (often known simply as the Mastiff)
perhaps descended from the ancient Alaunt and Pugnaces Britanniae, with a significant input from the
Alpine Mastiff in the 19th century. Distinguishable by enormous size, massive head, and a limited range
of colours, but always displaying a black mask, the Mastiff is noted for its gentle and loving nature.
Initial Opinion: Based on the evidence provided, it is stated that the Landseer has a limited range of
colors, while the English Mastiff has a wider range. This statement is consistent with the evidence, which
mentions that the Mastiff is distinguishable by a limited range of colors. Therefore, I agree with the
factuality of the answer in the QA pair.
Skeptic Agent: First, let’s analyze the opinions of the previous agent. The previous agent states that the
Landseer has a limited range of colors, while the English Mastiff has a wider range. This statement is
based on the evidence provided, which mentions that the Mastiff is distinguishable by a limited range of
colors. However, the evidence does not explicitly mention the color range of the Landseer. Therefore, it is
not entirely accurate to claim that the Landseer has a limited range of colors based on the given evidence.
Regarding the factuality of the answer in the QA pair, it is important to note that the evidence does not
provide specific information about the color range of the Landseer. Therefore, we cannot definitively
conclude whether the Landseer has a limited range of colors or not based on the given evidence. Without
further information, it is difficult to determine the factuality of the answer in the QA pair. Considering
the above analysis, I would rate the error severity as 2 (Moderate Errors) because the previous agents
statement is partially supported by the evidence, but it is not entirely accurate due to the lack of specific
information about the Landseers color range.
Trust Agent: I agree with the previous agents analysis of the opinions. The previous agent correctly points
out that the evidence provided does not explicitly mention the color range of the Landseer. Therefore, it is
not entirely accurate to claim that the Landseer has a limited range of colors based on the given evidence.
However, the evidence does mention that the English Mastiff has a limited range of colors, which supports
the previous agents statement about the Mastiff having a wider range of colors compared to the Landseer.
Leader Agent: After analyzing the opinions of the previous agents, it is clear that both agents agree that
the evidence provided does not explicitly mention the color range of the Landseer. Therefore, it is not
entirely accurate to claim that the Landseer has a limited range of colors based on the given evidence.
However, the evidence does mention that the English Mastiff has a limited range of colors, which supports
the previous agents statement about the Mastiff having a wider range of colors compared to the Landseer.
Regarding the factuality of the answer in the QA pair, it is important to note that the evidence does not
provide specific information about the color range of the Landseer. Therefore, we cannot definitively
conclude whether the Landseer has a limited range of colors or not based on the given evidence. Without
further information, it is difficult to determine the factuality of the answer in the QA [Link]
the above analysis, I would rate the error severity as 2 (Moderate Errors) because the previous agents
statement is partially supported by the evidence, but it is not entirely accurate due to the lack of specific
information about the Landseers color range.

Table 10: An example of the debate process by GPT-3.5-Turbo

Question: Does the Landseer or English Mastiff have a limited range of colors?
Answer: The Landseer has a limited range of colours, while the English Mastiff has a wider range.
Evidence: It is not to be confused with a white and black Newfoundland, which is also often called
a [Link] English Mastiff is a breed of extremely large dog (often known simply as the Mastiff)
perhaps descended from the ancient Alaunt and Pugnaces Britanniae, with a significant input from the
Alpine Mastiff in the 19th century. Distinguishable by enormous size, massive head, and a limited range
of colours, but always displaying a black mask, the Mastiff is noted for its gentle and loving nature.
Initial Opinion: Based on the evidence provided, it is stated that the Landseer has a limited range of
colors, while the English Mastiff has a wider range. This statement is consistent with the evidence, which
mentions that the Mastiff is distinguishable by a limited range of colors. Therefore, I agree with the
factuality of the answer in the QA pair.
Skeptic Agent: The previous agents opinion seems to be slightly misleading. The evidence does not
explicitly state that the Landseer has a limited range of colors. It only mentions that the Landseer is often
confused with a white and black Newfoundland, which might imply a limited color range, but its not
definitive. On the other hand, the evidence does state that the English Mastiff has a limited range of colors,
contradicting the QA pairs claim that the English Mastiff has a wider range. Therefore, the factuality of
the answer in the QA pair is questionable.
Trust Agent: While I agree with the previous agents assessment that the evidence does not explicitly
state that the Landseer has a limited range of colors, I believe the implication of a limited color range
from the confusion with a white and black Newfoundland is a valid interpretation. However, I concur
with the previous agents observation that the evidence contradicts the QA pairs claim about the English
Mastiffs color range. The evidence clearly states that the English Mastiff has a limited range of colors,
which contradicts the QA pairs assertion of a wider range. Therefore, while the QA pairs statement about
the Landseer may be inferred from the evidence, the claim about the English Mastiff is factually incorrect
based on the provided evidence.
Leader Agent: The Trust agents interpretation of the Landseers color range being limited due to its
confusion with a white and black Newfoundland is a plausible inference, but its not explicitly stated in the
evidence. The Skeptic agents doubt about this inference is also valid as the evidence does not directly
support it. However, both agents agree that the QA pairs claim about the English Mastiff having a wider
color range is contradicted by the evidence, which states that the English Mastiff has a limited range of
colors. Therefore, while the claim about the Landseers color range may be subject to interpretation, the
claim about the English Mastiff is clearly incorrect based on the provided evidence.

Table 11: An example of the debate process by GPT-4

Claim: Does the Landseer or English Mastiff have a limited range of colors? The Landseer has a limited range of colours, while
the English Mastiff has a wider range.

Evidence: It is not to be confused with a white and black Newfoundland, which is also often called a landseer. The English
Mastiff is a breed of extremely large dog (often known simply as the Mastiff) perhaps descended from the ancient Alaunt and
Pugnaces Britanniae, with a significant input from the Alpine Mastiff in the 19th century. Distinguishable by enormous size,
massive head, and a limited range of colours, but always displaying a black mask, the Mastiff is noted for its gentle and loving
nature.

Initial state input: Initial [Link](Claim, Evidence)

Initial answer:
{
"Opinion": "Based on the evidence provided, it is stated that the Landseer has a limited range of colors, while the English
Mastiff has a wider range. This statement is consistent with the evidence, which mentions that the Mastiff is distinguishable by a
limited range of colors. Therefore, I agree with the factuality of the answer in the QA pair.",
"Factuality": True,
"Error severity": 0
}

Skeptic agent input: Skeptic prompt + "Here is the response from the previous agent:" + Initial answer
Skeptic agent output:
{
"Opinion": "The previous agents opinion seems to be slightly misleading. The evidence does not explicitly state that the Landseer
has a limited range of colors. It only mentions that the Landseer is often confused with a white and black Newfoundland, which
might imply a limited color range, but its not definitive. On the other hand, the evidence does state that the English Mastiff has a
limited range of colors, contradicting the QA pairs claim that the English Mastiff has a wider range. Therefore, the factuality of
the answer in the QA pair is questionable.",
"Factuality": False,
"Error severity": 4
}

Trust agent input: Trust prompt + "Here is the response from the previous agent:" + Skeptic agent output
Trust agent output:
{
"Opinion": "While I agree with the previous agents assessment that the evidence does not explicitly state that the Landseer
has a limited range of colors, I believe the implication of a limited color range from the confusion with a white and black
Newfoundland is a valid interpretation. However, I concur with the previous agents observation that the evidence contradicts the
QA pairs claim about the English Mastiffs color range. The evidence clearly states that the English Mastiff has a limited range of
colors, which contradicts the QA pairs assertion of a wider range. Therefore, while the QA pairs statement about the Landseer
may be inferred from the evidence, the claim about the English Mastiff is factually incorrect based on the provided evidence.",
"Factuality": False,
"Error severity": 4
}

Leader agent input: Leader prompt + "Here is the response from the Skeptic agent:" + Skeptic agent output + "Here is the
response from the Trust agent:" + Trust agent output
Leader agent output:
{
"Opinion": "The Trust agents interpretation of the Landseers color range being limited due to its confusion with a white and
black Newfoundland is a plausible inference, but its not explicitly stated in the evidence. The Skeptic agents doubt about this
inference is also valid as the evidence does not directly support it. However, both agents agree that the QA pairs claim about the
English Mastiff having a wider color range is contradicted by the evidence, which states that the English Mastiff has a limited
range of colors. Therefore, while the claim about the Landseers color range may be subject to interpretation, the claim about the
English Mastiff is clearly incorrect based on the provided evidence.",
"Factuality": False,
"Error severity": 4
}

Table 12: Detailed Inference Process