ArchRAG: Attributed Community-based Hierarchical

Retrieval-Augmented Generation
Shu Wang1 , Yixiang Fang1 , Yingli Zhou1 , Xilin Liu2 , Yuchi Ma2 ,
1
The Chinese University of Hong Kong, Shenzhen
2
Huawei Cloud Computing Technologies CO., LTD.
{shuwang3, yinglizhou}@[Link], fangyixiang@[Link], {liuxilin3, mayuchi1}@[Link]

Abstract
arXiv:2502.09891v3 [[Link]] 8 Aug 2025

Retrieval-Augmented Generation (RAG) has proven effective

in integrating external knowledge into large language models
(LLMs) for solving question-answer (QA) tasks. The state-
of-the-art RAG approaches often use the graph data as the
external data since they capture the rich semantic informa-
tion and link relationships between entities. However, exist-
ing graph-based RAG approaches cannot accurately identify
the relevant information from the graph and also consume
large numbers of tokens in the online retrieval process. To
address these issues, we introduce a novel graph-based RAG Figure 1: The general workflow of graph-based RAG, which
approach, called Attributed Community-based Hierarchical retrieves relevant information (e.g., nodes, subgraphs, or tex-
RAG (ArchRAG), by augmenting the question using at- tual information) to facilitate the LLM generation.
tributed communities, and also introducing a novel LLM-
based hierarchical clustering method. To retrieve the most rel-
evant information from the graph for the question, we build a
novel hierarchical index structure for the attributed communi- In the past year, a surge of graph-based RAG meth-
ties and develop an effective online retrieval method. Experi- ods (2025; 2024c; 2024; 2024; 2024; 2023) has emerged,
mental results demonstrate that ArchRAG outperforms exist- each proposing different retrieval strategies to extract de-
ing methods in both accuracy and token cost. tailed information for response generation. Among them,
GraphRAG (2024), proposed by Microsoft, is the most
Introduction prominent and the first to leverage community summariza-
tion for abstract QA. It builds a knowledge graph (KG)
Retrieval-Augmented Generation (RAG) has emerged as a from the external corpus, detects communities using Lei-
core approach for enhancing large language models (LLMs) den (2019), and generates a summary for each community
by enabling access to domain-specific and real-time updated using LLMs. For abstract questions that require high-level
knowledge beyond their pre-training corpus (2024; 2023; information, it adopts a Global Search approach, traversing
2024; 2024; 2024b; 2024; 2024). By improving the trustwor- all communities and using LLMs to retrieve the most rel-
thiness and interpretability of LLMs, RAG has been widely evant summaries. In contrast, for specific questions, it em-
adopted across a broad range of applications (2024a; 2024a; ploys a Local Search method to retrieve entities, relevant text
2024; 2024; 2023; 2024; 2024b). Current state-of-the-art chunks, and low-level communities, providing the multi-hop
RAG approaches often use graph-structured data as exter- detailed information for accurate answers.
nal knowledge, due to its ability to capture the rich seman- Although some methods claim that GraphRAG underper-
tics and relationships. Given a question Q, the key idea of forms and is difficult to apply in practice (2024; 2024a),
graph-based RAG is to retrieve relevant information (e.g., our re-examination shows that it is primarily constrained
nodes, subgraphs, or textual information) from the graph, by the following limitations (L): L1. Low community qual-
incorporate them with Q as the prompt, and feed them into ity: GraphRAG uses the Leiden (2019) algorithm to detect
the LLM, as illustrated in Figure 1. Several recent meth- communities, but this approach relies solely on graph struc-
ods (2024; 2024; 2024; 2024a; 2024c; 2024; 2024) address ture and ignores the rich semantics of nodes and edges. As
two common question answering (QA) tasks: abstract ques- a result, the detected communities often consist of differ-
tions, which require reasoning over high-level themes (e.g., ent themes, which leads to the poor quality of community
“What are the potential impacts of LLMs on education?”), summaries and further decreases its performance. L2. Lim-
and specific questions, which focus on entity-centric factual ited compatibility: While GraphRAG employs Global and
details (e.g., “Who won the Turing Award in 2024?”).
Copyright © 2026, Association for the Advancement of Artificial Intelligence ([Link]). All rights reserved.
Local Search strategies, each retrieves graph elements at state-of-the-art graph-based RAG methods on specific ques-
only one granularity, making it inadequate for simultane- tions and shows notable gains on abstract QA. Moreover,
ously addressing both abstract and specific questions and ArchRAG is very token-efficient, saving up to 250 times the
limiting its applicability in real-world open-ended scenar- token usage compared to GraphRAG (2024).
ios. L3. High generation cost: Although GraphRAG per- In summary, our main contributions are as follows:
forms well on abstract questions, analyzing all communities • We present a novel graph-based RAG approach by using
with LLMs is both time- and token-consuming. For exam- ACs that are organized hierarchically and detected by an
ple, GraphRAG detects 2,984 communities in the Multihop- LLM-based hierarchical clustering method.
RAG (2024) dataset, and answering just 100 questions in- • To index ACs, we propose a novel hierarchical index
curs a cost of approximately $650 and 106 million tokens1 , structure called C-HNSW and also develop an efficient
which is an impractical overhead. online retrieval method.
To tackle the above limitations of GraphRAG, in this
• Extensive experiments show that ArchRAG is both
paper, we propose a novel graph-based RAG approach,
highly effective and efficient, and achieves state-of-the-
called Attributed Community-based Hierarchical RAG
art performance on both abstract and specific QA tasks.
(ArchRAG). ArchRAG leverages attributed communities
(ACs) and introduces an efficient hierarchical retrieval strat-
egy to adaptively support both abstract and specific ques- Related Work
tions. To mitigate L1, we detect high-quality ACs by exploit- In this section, we review the related works, includ-
ing both links and the attributes of nodes, ensuring that each ing Retrieval-Augmentation-Generation (RAG) approaches,
AC comprises nodes that are not only densely connected but and LLMs for graph mining and learning.
also share similar semantic themes (2009). We further pro- • RAG approaches. RAG has been proven to excel
pose a novel LLM-based iterative framework for hierarchi- in many tasks, including open-ended question answer-
cal AC detection, which can incorporate any existing com- ing (2024; 2023), programming context (2024b; 2023),
munity detection methods (2009; 2019; 2007; 2007; 2016). SQL rewrite (2025; 2024), and data cleaning (2024; 2022;
In each iteration, we detect ACs based on both attribute sim- 2024). The naive RAG technique relies on retrieving query-
ilarity and connectivity, summarize each AC using an LLM, relevant contexts from external knowledge bases to mit-
and construct a higher-level graph by treating each AC as a igate the “hallucination” of LLMs. Recently, many RAG
node, connecting pairs with similar summaries. By iterating approaches (2025; 2024; 2024; 2024; 2024c; 2024; 2024;
the above steps multiple times, we obtain a set of ACs that 2024) have adopted graph structures to organize the infor-
can be organized into a hierarchical tree structure. mation and relationships within documents, leading to im-
To effectively address L2, we organize all ACs and en- proved performance. For more details, please refer to the re-
tities into a hierarchical index and retrieve relevant ele- cent survey of graph-based RAG methods (2024).
ments from all levels to support both abstract and specific • LLM for graph mining. Recent advances in LLMs
questions. Entities offer fine-grained details, while LLM- have offered opportunities to leverage LLMs in graph min-
generated AC summaries capture relational structures and ing. These include using LLMs for KG construction (2024),
provide high-level condensed overviews (2024; 2018), mak- addressing complex graph mining tasks (2024a; 2024; 2024;
ing them suitable for both multi-hop reasoning and abstract 2024b), and employing KG to enhance the LLM reason-
insight extraction. To support efficient retrieval across levels, ing (2023; 2024c; 2023; 2023; 2024; 2023; 2025). For in-
we propose C-HNSW (Community-based HNSW), a novel stance, RoG (2023) proposes a planning-retrieval-reasoning
hierarchical index inspired by the HNSW algorithm (2018) framework that retrieves reasoning paths from KGs to guide
for approximate nearest neighbor (ANN) search. LLMs conducting faithful reasoning. StructGPT (2023) and
To mitigate the high generation cost caused by travers- ToG (2023) treat LLMs as agents that interact with KGs to
ing all communities (L3), we propose a hierarchical search find reasoning paths leading to the correct answers.
with adaptive filtering to efficiently select the most relevant
ACs and entities while maintaining performance. Specifi- Our Approach ArchRAG
cally, we design an efficient hierarchical retrieval algorithm We begin by presenting the overall workflow and design ra-
over the proposed C-HNSW index, which supports top-k tionale of ArchRAG, followed by detailed descriptions of
nearest neighbor search across multiple levels, thereby fa- each component. As illustrated in Figure 2, our proposed
cilitating access to multi-level relevant information. Further- ArchRAG consists of two phases. In the offline indexing
more, the adaptive filtering mechanism identifies the most phase, ArchRAG first constructs a KG from the corpus, then
informative results at each level, making the retrieved infor- detects ACs by a novel LLM-based hierarchical clustering
mation complementary. method, and finally builds the C-HNSW index. During the
We have extensively evaluated ArchRAG on real-world online retrieval phase, ArchRAG first converts the question
datasets, and the results show that it consistently outper- into a query vector, then retrieves relevant information from
forms existing methods in both abstract and specific QA the C-HNSW index, and finally generates answers through
tasks. ArchRAG achieves a 10% higher accuracy than an adaptive filtering-based generation process
Our ArchRAG detects ACs by exploiting both links and
1 attributes, and organizes ACs and entities into a novel hierar-
The cost of GPT-4o is $10/M tokens for output and $2.50/M
tokens for input (for details, please refer to OpenAI pricing). chical index, C-HNSW, yielding the following advantages:
Figure 2: ArchRAG consists of two phases: offline indexing and online retrieval. For the online retrieval phase, we show an
example of using ArchRAG to answer a question in the HotpotQA dataset.

1) Each AC group densely connected entities with shared and SCAN (2007)). Specifically, we propose to augment the
themes and a high-quality summary. 2) The hierarchical KG by linking entities if their attribute similarities are larger
structure captures multiple levels of abstraction: lower-level than a threshold, and then associate each pair of linked en-
entities and communities encode detailed KG knowledge, tities with a weight denoting their attribute similarity value.
while higher-level communities provide global context, en- Next, we generate the ACs using any given graph clustering
abling ArchRAG to address questions at varying granular- algorithm. In this way, both node attributes and structural
ity. and 3) The C-HNSW index efficiently retrieves relevant links are jointly considered during community detection.
information across levels, supporting fast and accurate re- Algorithm 1 shows the above iterative clustering process.
sponses to both abstract and specific questions. Given a graph augmentation method Aug, clustering algo-
rithm GCluster and stopping condition T , we perform the
Offline Indexing following steps in each iteration: (1) augmenting the graph
KG construction. ArchRAG builds a KG by prompting (line 3); (2) computing the edge weights (lines 4-5); (3) clus-
the LLM to extract entities and relations from each chunk tering the augmented graph (line 6); (4) generating a sum-
of the text corpus D. Specifically, all text contexts are seg- mary for each community using LLM (line 7); and (5) build-
mented into chunks based on specified chunk length, en- ing a new attributed graph where each node denotes an AC
abling the LLM to extract entities and relations from each and two nodes are linked if their community members are
chunk using in-context learning (2020), thus forming sub- connected (line 9). We repeat the iterations until the stop-
graphs. These subgraphs are then merged, with entities and ping condition T (such as insufficient nodes or reaching the
relations that appear repeatedly across multiple subgraphs specified level limit) is met. Since each iteration corresponds
being consolidated by the LLM to generate a complete de- to one layer, all the ACs HC can be organized into a multi-
scription. Finally, we get a KG, denoted by G(V, E), where layer hierarchical tree structure, denoted by ∆, where each
V and E are sets of vertices and edges, respectively, and community in one layer includes multiple communities in
each vertex and edge is associated with textual attributes. the next layer. Appendix also provides more details.

LLM-based hierarchical clustering. We propose an it- C-HNSW index. Given a query, to efficiently identify the
erative LLM-based hierarchical clustering framework that most relevant information from each layer of the hierarchical
supports arbitrary graph augmentation (e.g., KNN connec- tree ∆, a naive method is to build a vector database for the
tions and CODICIL (2013)) and graph clustering algorithms ACs in each layer, which is costly in both time and space.
(e.g., weighted Leiden (2019), weighted spectral clustering, To tackle this issue, we propose to build a single hierar-
 Algorithm 1: LLM-based hierarchical clustering Intuitively, since the two types of links above are es-
tablished based on nearest neighbors, C-HNSW allows us
input : G(V, E), Aug, GCluster, T
1 T ← False, HC ← ∅; to quickly search the relevant information for a query by
2 repeat traversing along with these links. Note that C-HNSW is dif-
3 G′ (V, E ′ ) ← Aug(G(V, E)); ferent from HNSW since it has intra-layer links and each
4 for each e′ = (u, v) ∈ E ′ do node exists in only one layer.
5 update the weight of e′ as 1 − cos(zu , zv ); • The construction of C-HNSW. We propose a top-down
6 C ← GCluster(G′ (V, E ′ )); approach to build a C-HNSW. Specifically, by leveraging the
7 for each c ∈ C do generate summary of c by LLM ; query process of C-HNSW, which will be introduced in on-
8 HC ← HC ∪ C; line retrieval, nodes are progressively inserted into the in-
9 G(V, E) ← build a new graph using C and E ′ ; dex starting from the top layer, connecting intra-layer links
// update T according to G(V, E); within the same layer and updating the inter-layer links. For
10 until T =True; lack of space, we give the details of the construction algo-
11 return HC; rithm in the appendix.

Online retrieval
chical index for all the communities. Recall that the ACs In the online retrieval phase, after obtaining the query vector
in ∆ form a tree structure, and the number of nodes de- for a given question, ArchRAG generates the final answer by
creases as the layer level increases. Since this tree struc- first conducting a hierarchical search on the C-HNSW index
ture is similar to the HNSW (Hierarchical Navigable Small and then analyzing and filtering the retrieved information.
World) index which is the most well-known index for ef-
ficient ANN search (2018), we propose to map entities and Hierarchical search. We propose an efficient and fast re-
ACs of ∆ into high-dimensional nodes, and then build a uni- trieval algorithm, hierarchical search, to retrieve nodes from
fied Community-based HNSW (C-HNSW) index for them. each layer of the C-HNSW structure. Intuitively, retrieving
nodes from a given layer in C-HNSW requires starting from
• The structure of C-HNSW. Conceptually, the C-
the top layer and searching downward through two types
HNSW index is a list of simple graphs with links between
of links (i.e., intra-layer and inter-layer links) to locate the
them, denoted by H = (G, Linter ) with G = {G0 =
nearest neighbors at the given layer. In contrast, our hierar-
(V0 , E0 ), G1 = (V1 , E1 ), · · · , GL = (VL , EL )}, where Gi
chical search algorithm accelerates this process by reusing
is a simple graph and each node of the simple graph cor-
intermediate results, the nearest neighbors found in higher
responds to an attributed community or entity. The number
layers, as the starting node for lower layers. This approach
L of layers in H is the same as that of ∆. Specifically, for
avoids redundant computations that would otherwise arise
each attributed community or entity in the i-th layer of ∆,
from repeatedly searching from the top layer, thereby en-
we map it to a high-dimensional node in the i-th layer of H
abling efficient multi-layer retrieval.
by using a language model (e.g., nomic-embed-text (2024)).
Algorithm 2 illustrates hierarchical search. Given the C-
We next establish two types of links between these high-
HNSW H, query point q, and the number k of nearest neigh-
dimensional nodes, i.e., intra-layer and inter-layer links:
bors to retrieve at each layer, the hierarchical search algo-
• Intra-layer links: These links exist between nodes in the rithm can be implemented by the following iterative process:
same layers. Specifically, for each node in each layer, we
1. Start from a random node at the highest layer L, which
link it to at least M nearest neighbors within the same
serves as the starting node for layer L (line 1).
layer, where M is a predefined value, and the nearest
neighbors are determined according to a given distance 2. For each layer i from the top layer L down to layer 0,
metric d. Thus, all the intra-layer links are edges in all the algorithm begins at the starting node and performs a
SL
the simple graphs: Lintra = i=0 Ei . greedy traversal (i.e., the SearchLayer procedure) to
find the set Ri of the k nearest neighbors of q. The set Ri
• Inter-layer links: These links cross two adjacent lay- is then merged into the final result set R (lines 4–5).
ers. Specifically, we link each node in each layer to
its nearest neighbor in the next layer. As a result, all 3. The closest neighbor c of q is then obtained from Ri , and
the inter-layer links can be represented as Linter = the algorithm proceeds to the next layer by traversing the
SL inter-layer link of c, using it as the starting node for the
i=1 {(v, ψ(v))|v ∈ Vi , ψ(v) ∈ Vi−1 }, where ψ(·) : subsequent search (lines 6–7).
Vi → Vi−1 is the injective function that identifies the
nearest neighbor of each node in the lower layer. Specifically, the greedy traversal strategy compares the
distance between the query point and the visited nodes dur-
For example, in Figure 2, the C-HNSW index has three
ing the search process. It maintains a candidate expansion
layers (simple graphs), incorporating all the ACs. Within
queue Q and a dynamic nearest neighbor set K containing
each layer, each node is connected to its two nearest neigh-
k elements, along with a stopping condition:
bors via intra-layer links, denoted by undirected edges. The
inter-layer links are represented by arrows, e.g., the green • Expansion Queue Q: For each neighbor x of a visited
community at layer L1 is connected to the green entity (its node, if d(x, q) < d(f, q), where f is the furthest node
nearest neighbor at layer L0 ). from R to q, then x is added to the expansion queue.
 Algorithm 2: Hierarchical search limit being truncated. This process can be represented as:
input : H = (G, Linter ), q, k. Ai = LLM (Pf ilter ||Ri ) (1)
1 s ← a random node in the highest layer L; Output = LLM (Pmerge ||Sort({A0 , A1 , · · · , An })) (2)
2 R ← ∅;
3 for i ← L, · · · , 0 do where Pf ilter and Pmerge represent the prompts for extract-
4 Ri ← SearchLayer (Gl = (Vl , El ), q, s, k); ing relevant information and summarizing, respectively, Ai ,
5 R ← R ∪ Ri ; i ∈ 0 · · · n denotes the filtered analysis report. The sort func-
6 c ← get the nearest node from Ri ; tion orders the content based on the relevance scores from
7 s ← find the node in layer i − 1 via the inter-layer the analysis report.
links of c;
8 return R; Experiments
9 Procedure SearchLayer(Gi = (Vi , Ei ), q, s, k):
10 V ← {s}, K ← {s}, Q ← initialize a queue In this section, we conduct a comprehensive evaluation of
containing s; our ArchRAG, focusing on both efficiency and performance.
11 while |K| > 0 do
12 c ← nearest node in Q; Setup
13 f ← furthest node in K;
14 if d(c, q) > d(f, q) then break ; Table 1: Datasets used in our experiments. Acc, Rec, Blue,
15 for each neighbor x ∈ N (c) in Gi do Met, and Rou denote Accuracy, Recall, BLEU-1, METEOR,
16 if x ∈ V then continue; and ROUGE-L F1.
17 V ← V ∪ {x};
18 f ← furthest node in K;
19 if d(x, q) < d(f, q) or |K| < k then Dataset Multihop-RAG HotpotQA NarrativeQA
20 Q ← Q ∪ {x}, K ← K ∪ {x}; Passages 609 9,221 1,572
21 if |K| > k then remove f from K; Tokens 1,426,396 1,284,956 121,152,448
22 return K; Nodes 23,353 37,436 650,571
Edges 30,716 30,758 679,426
Questions 2,556 1,000 43,304
Metrics Acc, Rec Acc, Rec Blue, Met, Rou
• Dynamic Nearest Neighbor Set K: Nodes added to C are
used to update K, ensuring that it maintains no more than
Datasets. We evaluate ArchRAG on both specific and
k elements, where k is the number of query results.
abstract QA tasks. For specific QA, we use Multihop-
• Stopping Condition: The traversal terminates if a node x RAG (2024), HotpotQA (2018), and NarrativeQA (2018),
expanded from Q satisfies d(n, q) > d(n, f ), where f is all of which are extensively utilized within the QA and
the furthest node in K from the query point q. Graph-based RAG research communities (2022; 2024;
2022; 2024; 2024; 2023). For abstract QA, we follow the
After completing the hierarchical search and obtaining the GraphRAG (2024) method and reuse the Multihop-RAG
ACs and entities from each layer, we further extract their corpus, prompting LLM to generate questions that convey
associated textual information. In particular, at the bottom a high-level understanding of dataset contents. The statistics
layer, we also extract the relationships between the retrieved of these datasets are reported in Table 1.
entities, resulting in the textual subgraph representation de-
Baselines. Our experiments consider three configurations:
noted as R0 . We denote all the retrieved textual information
from each layer as Ri , where i ∈ 0, 1, . . . , L, which will be • Inference-only: Using an LLM to answer questions with-
used in the adaptive filtering-based generation process. out retrieval, i.e., Zero-Shot and CoT (2022).
• Retrieval-only: Retrieval models extract relevant chunks
Adaptive filtering-based generation. While some opti- from all documents and use them as prompts for LLMs.
mized LLMs support longer text inputs, they may still en- We select strong and widely used retrieval models: BM25
counter issues such as the “lost in the middle” dilemma (Liu (1994) and Vanilla RAG.
et al. 2024b). Thus, direct utilization of retrieved informa- • Graph-based RAG: These methods leverage graph data
tion comprising multiple text segments for LLM-based an- during retrieval. We select RAPTOR (2024), HippoRAG
swer generation risks compromising output accuracy. (2024), GraphRAG (2024), and LightRAG (2024). Partic-
To mitigate this limitation, we propose an adaptive ularly, GraphRAG has two versions, i.e., GGraphRAG and
filtering-based method that harnesses the LLM’s inherent LGraphRAG, which use global and local search methods,
reasoning capabilities. We first prompt the LLM to extract respectively. Similarly, LightRAG integrates local search,
and generate an analysis report from the retrieved informa- global search, and hybrid search, denoted by LLightRAG,
tion, identifying the parts that are most relevant to answer- HLightRAG, and HyLightRAG, respectively.
ing the query and assigning relevance scores to these reports. In GGraphRAG, all communities below the selected level
Then, all analysis reports are integrated and sorted, ensuring are first retrieved, and then the LLM is used to filter out ir-
that the most relevant content is used to summarize the final relevant communities. This process can be viewed as uti-
response to the query, with any content exceeding the text lizing the LLM as a retriever to find relevant communities
 VR LR C1 C2 AR VR LR C1 C2 AR VR LR C1 C2 AR VR LR C1 C2 AR
VR 50 46 18 18 1 VR 50 64 3 12 8 VR 50 39 15 21 8 VR 50 46 14 18 4
LR 54 50 21 29 16 LR 36 50 52 63 33 LR 61 50 59 30 35 LR 54 50 48 31 22
C1 82 79 50 86 18 C1 97 48 50 52 46 C1 85 41 50 60 42 C1 86 52 50 70 31
C2 82 71 14 50 16 C2 88 37 48 50 42 C2 79 70 40 50 38 C2 82 69 30 50 30
AR 99 84 82 84 50 AR 92 67 54 58 50 AR 92 65 58 62 50 AR 96 78 69 70 50
(a) Comprehensiveness (b) Diversity (c) Empowerment (d) Overall

Figure 3: Head-to-head win rates for abstract QA, comparing each row method against each column (higher is better). VR, LR,
and AR denote Vanilla RAG, HyLightRAG, and ArchRAG, respectively.

within the corpus. According to the selected level of commu- stantial performance advantage over other baseline meth-
nities (2024), GGraphRAG can be further categorized into ods on these datasets. The experimental results suggest that
C1 and C2, representing high-level and intermediate-level not all communities are suitable for specific QA tasks, as
communities, respectively, with C2 as the default. the GGraphRAG performs poorly. Furthermore, GraphRAG
does not consider node attributes during clustering, which
Metrics & Implementation. For the specific QA tasks, causes the community’s summary to become dispersed,
we use Accuracy and Recall to evaluate performance on making it difficult for the LLM to extract relevant informa-
the first two datasets based on whether gold answers are in- tion from a large number of communities. Thus, we gain an
cluded in the generations instead of strictly requiring exact interesting insight: LLM may not be a good retriever, but is a
matching, following (2024; 2022; 2023). We also use the good analyzer. We further analyze the reasons behind the un-
official metrics of BLEU, METEOR, and ROUGE-l F1 in derperformance of each graph-based RAG method and sup-
the NarrativeQA dataset. For the abstract QA task, we fol- port our claims with empirical evidence in the appendix.
low prior work (2024) and adopt a head-to-head compari-
son approach using an LLM evaluator (GPT-4o). Overall,
we utilize four evaluation dimensions: Comprehensiveness,
Zero-Shot CoT BM25 Vanilla RAG
Diversity, Empowerment, and Overall. For implementation,
RAPTOR HippoRAG LLightRAG HLightRAG
we mainly use Llama 3.1-8B (2024) as the default LLM and
use nomic-embed-text (2024) as the text embedding model. HyLightRAG LGraphRAG GGraphRAG ArchRAG
We use KNN for graph augmentation and the weighted Lei-
105
time (s)

den algorithm for community detection. For retrieval item

k, we search the same number of items at each layer, with 104

N/A
k = 5 as the default. All methods are required to complete 103
index construction and query execution within 3 days, re- 102 Multihop-RAG HotpotQA
spectively. Additional details are provided in the appendix, (a) Time cost
and our codes are provided in the supplementary material.
1032
token (M)

101
Overall results 100

N/A
10
We compare our method with baseline methods in solving
Multihop-RAG HotpotQA
both abstract and specific QA tasks.
• Results of abstract QA tasks. We compare ArchRAG (b) Token cost
against baselines across four dimensions on the Multihop-
RAG dataset. For the LightRAG, we only compare the Hy- Figure 4: Comparison of query efficiency.
LightRAG method, as it represents the best version (2024).
As shown in Figure 3, GGraphRAG outperforms other base-
line methods, while our method achieves comparable perfor- • Efficiency of ArchRAG. We compare the time cost and
mance on the diversity and empowerment dimensions and token usage of ArchRAG with those of other baseline meth-
significantly surpasses it on the comprehensive dimension. ods. As shown in Figure 4, ArchRAG demonstrates signifi-
Overall, by leveraging ACs, ArchRAG demonstrates supe- cant time and cost efficiency for online queries. For example,
rior performance in addressing abstract QA tasks. token usage on the HotpotQA dataset is cut by 250× with
• Results of specific QA tasks. Table 2 reports the ArchRAG compared to GraphRAG-Global, from 1,394M
performance of each method on three datasets. Note that tokens down to 5.1M tokens.
GGraphRAG fails to complete querying on the NarrativeQA To further evaluate ArchRAG, we test the efficiency of
dataset within the 3-day time limit. RAPTOR is unable hierarchical search, indexing performance, and effective-
to build the index on datasets like HotpotQA, which con- ness on an additional dataset, RAG-QA Arena (2024). Re-
tains a large number of text chunks. Its Gaussian Mix- sults show that ArchRAG achieves up to 5.4× faster retrieval
ture Model (GMM) clustering algorithm requires prohibitive than basic HNSW, maintains efficient indexing, and achieves
computational time and suffers from non-termination issues state-of-the-art performance on the RAG-QA Arena dataset.
during clustering. Clearly, ArchRAG demonstrates a sub- Additional details are provided in the appendix.
Table 2: Performance comparison of different methods across various datasets for solving specific QA tasks. The best and
second-best results are marked in bold and underlined. OOT: Not finished within 3 days.

Multihop-RAG HotpotQA NarrativeQA

Baseline Type Method
(Accuracy) (Recall) (Accuracy) (Recall) (BLEU-1) (METEOR) (ROUGE-L F1)
Zero-shot 47.7 23.6 28.0 31.8 8.0 7.9 8.6
Inference-only
CoT 54.5 28.7 32.5 39.7 5.0 8.1 6.4
BM25 37.6 19.4 49.7 53.6 2.0 4.9 2.8
Retrieval-only
Vanilla RAG 58.6 31.4 50.6 56.1 2.0 4.9 2.8
RAPTOR 59.1 34.1 N/A N/A 5.5 12.5 9.1
HippoRAG 38.9 19.1 51.3 56.8 2.2 5.0 2.8
LLightRAG 44.1 25.1 34.1 41.8 4.5 8.7 6.6
Graph-based RAG
HLightRAG 48.5 28.7 25.6 33.3 4.4 8.1 6.1
HyLightRAG 50.3 30.3 35.6 43.3 5.0 9.4 7.0
LGraphRAG 40.1 23.8 29.7 35.5 3.9 3.3 3.5
GGraphRAG 45.9 28.4 33.5 42.6 OOT OOT OOT
Our proposed ArchRAG 68.8 37.2 65.4 69.2 11.5 15.6 17.6

Detailed Analysis Table 3: Comparing the performance of different variants

of ArchRAG on the specific QA tasks. Acc and Rec denote
To better understand the effectiveness of our proposed
Accuracy and Recall, respectively.
ArchRAG, we perform the following ablation study and ex-
periment with a GGraphRAG variant.
• Ablation study. To evaluate the contributions of Multihop-RAG HotpotQA
Method variants
different components, we design several ArchRAG vari- (Acc) (Rec) (Acc) (Rec)
ants and conduct ablation experiments. These include ArchRAG 68.8 37.2 65.4 69.2
two modifications to the LLM-based hierarchical clus- - Spec. 67.1 36.7 60.5 63.7
tering framework, three targeting core design elements - Spec. (No Aug) 62.8 34.2 64.8 63.2
in ArchRAG—attributes, hierarchy, and communities—and - Leiden 63.2 34.1 61.7 64.8
one direct prompting variant, as detailed below: - Single-Layer 63.8 36.4 60.1 63.6
• Spec.: Spectral clustering instead of weighted Leiden. - Entity-Only 61.2 34.7 59.9 63.1
- Direct Prompt 59.9 29.6 40.7 45.4
• Spec. (No Aug): Spectral clustering without graph aug-
mentation.
Table 4: Results of GraphRAG variants using our ACs.
• Leiden: Replaces our clustering framework with Leiden.
• Single-Layer: Replaces our hierarchical index and search Multihop-RAG HotpotQA
with a single-layer community. Method
(Acc) (Rec) (Acc) (Rec)
• Entity-Only: Generate the response using entities only.
• Direct Prompt: Direct prompts the LLM to generate the GGraphRAG 45.9 28.4 33.5 42.6
response without the adaptive filtering-based generation. GraphRAG+AC 49.3 31.4 50.6 (51.0% ↑) 52.8
ArchRAG 68.8 37.2 65.4 69.2
As shown in Table 3, the performance of ArchRAG on
specific QA tasks decreases when each feature is removed,
with the removal of the community component resulting
community quality of its LLM-based hierarchical cluster-
in the most significant drop. Additionally, the direct vari-
ing. ArchRAG consistently achieves state-of-the-art perfor-
ant demonstrates that the adaptive filtering-based generation
mance across all backbones, remains robust under vary-
process can effectively extract relevant information from re-
ing top-k values, and demonstrates high-quality community
trieved elements.
detection. We also provide additional case studies of our
• Impact of Attributed Communities in RAG. We
ArchRAG, which illustrate the detailed workflow. Detailed
propose a new variant, GraphRAG+AC, which replaces the
experiments are provided in the appendix.
original Leiden-based communities in GraphRAG with our
ACs, while preserving the original Global Search pipeline.
As shown in Table 4, this variant results in a significant per- Conclusion
formance improvement compared to the original approach. In this paper, we propose ArchRAG, a novel graph-based
Specifically, on the HotpotQA dataset, GraphRAG+AC im- RAG approach, by augmenting the question using attributed
proves accuracy by 51% compared to GGraphRAG. communities from the knowledge graph built on the external
We further test ArchRAG under different LLM back- corpus, and building a novel index for efficient retrieval of
bones, various top-k retrieval settings, and evaluate the relevant information. Our experiments show that ArchRAG
is highly effective and efficient. In the future, we will explore Grover, A.; and Leskovec, J. 2016. node2vec: Scalable fea-
fast parallel graph-based RAG methods to process large- ture learning for networks. In Proceedings of the 22nd ACM
scale external corpus. SIGKDD international conference on Knowledge discovery
and data mining, 855–864.
References Guo, Z.; Xia, L.; Yu, Y.; Ao, T.; and Huang, C. 2024. Ligh-
Angelidis, S.; and Lapata, M. 2018. Summarizing opinions: tRAG: Simple and Fast Retrieval-Augmented Generation.
Aspect extraction meets sentiment prediction and they are arXiv e-prints, arXiv–2410.
both weakly supervised. arXiv preprint arXiv:1808.08858. Gutiérrez, B. J.; Shu, Y.; Gu, Y.; Yasunaga, M.; and Su,
Asai, A.; Wu, Z.; Wang, Y.; Sil, A.; and Hajishirzi, H. 2023. Y. 2024. HippoRAG: Neurobiologically Inspired Long-
Self-rag: Learning to retrieve, generate, and critique through Term Memory for Large Language Models. arXiv preprint
self-reflection. arXiv preprint arXiv:2310.11511. arXiv:2405.14831.
Brown, T. B. 2020. Language models are few-shot learners. Han, R.; Zhang, Y.; Qi, P.; Xu, Y.; Wang, J.; Liu, L.; Wang,
arXiv preprint arXiv:2005.14165. W. Y.; Min, B.; and Castelli, V. 2024. RAG-QA Arena: Eval-
Caliński, T.; and Harabasz, J. 1974. A dendrite method for uating Domain Robustness for Long-form Retrieval Aug-
cluster analysis. Communications in Statistics-theory and mented Question Answering. In Proceedings of the 2024
Methods, 3(1): 1–27. Conference on Empirical Methods in Natural Language
Cao, Y.; Han, S.; Gao, Z.; Ding, Z.; Xie, X.; and Zhou, S. K. Processing, 4354–4374.
2024. Graphinsight: Unlocking insights in large language He, X.; Tian, Y.; Sun, Y.; Chawla, N. V.; Laurent, T.; LeCun,
models for graph structure understanding. arXiv preprint Y.; Bresson, X.; and Hooi, B. 2024. G-retriever: Retrieval-
arXiv:2409.03258. augmented generation for textual graph understanding and
Charikar, M. S. 2002. Similarity estimation techniques from question answering. arXiv preprint arXiv:2402.07630.
rounding algorithms. In Proceedings of the thiry-fourth an- Hu, Y.; and Lu, Y. 2024. Rag and rau: A survey on retrieval-
nual ACM symposium on Theory of computing, 380–388. augmented language model in natural language processing.
Chen, N.; Li, Y.; Tang, J.; and Li, J. 2024a. Graphwiz: An arXiv preprint arXiv:2404.19543.
instruction-following language model for graph computa- Hu, Z.; Xu, Y.; Yu, W.; Wang, S.; Yang, Z.; Zhu, C.; Chang,
tional problems. In KDD. K.-W.; and Sun, Y. 2022. Empowering language models
Chen, S.; He, Y.; Cui, W.; Fan, J.; Ge, S.; Zhang, H.; Zhang, with knowledge graph reasoning for question answering.
D.; and Chaudhuri, S. 2024b. Auto-Formula: Recommend arXiv preprint arXiv:2211.08380.
Formulas in Spreadsheets using Contrastive Learning for Ta- Huang, Y.; and Huang, J. 2024. A Survey on Retrieval-
ble Representations. Proceedings of the ACM on Manage- Augmented Text Generation for Large Language Models.
ment of Data, 2(3): 1–27. arXiv preprint arXiv:2404.10981.
Chen, S.; Tang, N.; Fan, J.; Yan, X.; Chai, C.; Li, G.; and
Huang, Y.; Zhang, S.; and Xiao, X. 2025. KET-RAG:
Du, X. 2023. Haipipe: Combining human-generated and
A Cost-Efficient Multi-Granular Indexing Framework for
machine-generated pipelines for data preparation. Proceed-
Graph-RAG. arXiv preprint arXiv:2502.09304.
ings of the ACM on Management of Data, 1(1): 1–26.
Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Jeong, S.; Baek, J.; Cho, S.; Hwang, S. J.; and Park, J. C.
Letman, A.; Mathur, A.; Schelten, A.; Yang, A.; Fan, A.; 2024. Adaptive-rag: Learning to adapt retrieval-augmented
et al. 2024. The llama 3 herd of models. arXiv preprint large language models through question complexity. arXiv
arXiv:2407.21783. preprint arXiv:2403.14403.
Edge, D.; Trinh, H.; Cheng, N.; Bradley, J.; Chao, A.; Mody, Jiang, J.; Zhou, K.; Dong, Z.; Ye, K.; Zhao, W. X.; and Wen,
A.; Truitt, S.; and Larson, J. 2024. From local to global: A J.-R. 2023. Structgpt: A general framework for large lan-
graph rag approach to query-focused summarization. arXiv guage model to reason over structured data. arXiv preprint
preprint arXiv:2404.16130. arXiv:2305.09645.
Fan, W.; Ding, Y.; Ning, L.; Wang, S.; Li, H.; Yin, D.; Chua, Kočiskỳ, T.; Schwarz, J.; Blunsom, P.; Dyer, C.; Hermann,
T.-S.; and Li, Q. 2024. A survey on rag meeting llms: To- K. M.; Melis, G.; and Grefenstette, E. 2018. The narrativeqa
wards retrieval-augmented large language models. In Pro- reading comprehension challenge. Transactions of the As-
ceedings of the 30th ACM SIGKDD Conference on Knowl- sociation for Computational Linguistics, 6: 317–328.
edge Discovery and Data Mining, 6491–6501. Kojima, T.; Gu, S. S.; Reid, M.; Matsuo, Y.; and Iwasawa,
Gao, Y.; Xiong, Y.; Gao, X.; Jia, K.; Pan, J.; Bi, Y.; Dai, Y. 2022. Large language models are zero-shot reason-
Y.; Sun, J.; and Wang, H. 2023. Retrieval-augmented gen- ers. Advances in neural information processing systems, 35:
eration for large language models: A survey. arXiv preprint 22199–22213.
arXiv:2312.10997. Li, D.; Yang, S.; Tan, Z.; Baik, J. Y.; Yun, S.; Lee, J.; Chacko,
Ghimire, A.; Prather, J.; and Edwards, J. 2024. Gen- A.; Hou, B.; Duong-Tran, D.; Ding, Y.; et al. 2024. DALK:
erative AI in Education: A Study of Educators’ Aware- Dynamic Co-Augmentation of LLMs and KG to answer
ness, Sentiments, and Influencing Factors. arXiv preprint Alzheimer’s Disease Questions with Scientific Literature.
arXiv:2403.15586. arXiv preprint arXiv:2405.04819.
Li, Y.; Wang, S.; Ding, H.; and Chen, H. 2023. Large lan- Unified Framework for Data Manipulation with Large Lan-
guage models in finance: A survey. In Proceedings of the guage Models. Proceedings of Machine Learning and Sys-
fourth ACM international conference on AI in finance, 374– tems, 6: 465–482.
382. Robertson, S. E.; and Walker, S. 1994. Some simple effec-
Li, Z.; Yuan, H.; Wang, H.; Cong, G.; and Bing, L. 2025. tive approximations to the 2-poisson model for probabilistic
LLM-R2: A Large Language Model Enhanced Rule-based weighted retrieval. In SIGIR’94: Proceedings of the Sev-
Rewrite System for Boosting Query Efficiency. Proceedings enteenth Annual International ACM-SIGIR Conference on
of the VLDB Endowment, 1(18): 53–65. Research and Development in Information Retrieval, organ-
Liu, L.; Yang, X.; Lei, J.; Liu, X.; Shen, Y.; Zhang, Z.; ised by Dublin City University, 232–241. Springer.
Wei, P.; Gu, J.; Chu, Z.; Qin, Z.; et al. 2024a. A Survey Ruan, Y.; Fuhry, D.; and Parthasarathy, S. 2013. Efficient
on Medical Large Language Models: Technology, Applica- community detection in large networks using content and
tion, Trustworthiness, and Future Directions. arXiv preprint links. In Proceedings of the 22nd international conference
arXiv:2406.03712. on World Wide Web, 1089–1098.
Liu, N. F.; Lin, K.; Hewitt, J.; Paranjape, A.; Bevilacqua, Sarthi, P.; Abdullah, S.; Tuli, A.; Khanna, S.; Goldie, A.;
M.; Petroni, F.; and Liang, P. 2024b. Lost in the middle: and Manning, C. D. 2024. Raptor: Recursive abstrac-
How language models use long contexts. Transactions of the tive processing for tree-organized retrieval. arXiv preprint
Association for Computational Linguistics, 12: 157–173. arXiv:2401.18059.
Luo, L.; Li, Y.-F.; Haffari, G.; and Pan, S. 2023. Reasoning Schick, T.; Dwivedi-Yu, J.; Dessı̀, R.; Raileanu, R.; Lomeli,
on graphs: Faithful and interpretable large language model M.; Hambro, E.; Zettlemoyer, L.; Cancedda, N.; and
reasoning. arXiv preprint arXiv:2310.01061. Scialom, T. 2024. Toolformer: Language models can teach
themselves to use tools. Advances in Neural Information
Ma, S.; Xu, C.; Jiang, X.; Li, M.; Qu, H.; Yang, C.; Mao, Processing Systems, 36.
J.; and Guo, J. 2024. Think-on-Graph 2.0: Deep and Faith-
ful Large Language Model Reasoning with Knowledge- Siriwardhana, S.; Weerasekera, R.; Wen, E.; Kaluarachchi,
guided Retrieval Augmented Generation. arXiv preprint T.; Rana, R.; and Nanayakkara, S. 2023. Improving the do-
arXiv:2407.10805. main adaptation of retrieval augmented generation (RAG)
models for open domain question answering. Transactions
Malkov, Y. A.; and Yashunin, D. A. 2018. Efficient and ro- of the Association for Computational Linguistics, 11: 1–17.
bust approximate nearest neighbor search using hierarchical
navigable small world graphs. IEEE transactions on pattern Sun, J.; Xu, C.; Tang, L.; Wang, S.; Lin, C.; Gong, Y.; Shum,
analysis and machine intelligence, 42(4): 824–836. H.-Y.; and Guo, J. 2023. Think-on-graph: Deep and respon-
sible reasoning of large language model with knowledge
Mallen, A.; Asai, A.; Zhong, V.; Das, R.; Khashabi, D.; and graph. arXiv preprint arXiv:2307.07697.
Hajishirzi, H. 2022. When not to trust language models:
Sun, Z.; Zhou, X.; and Li, G. 2024. R-Bot: An LLM-based
Investigating effectiveness of parametric and non-parametric
Query Rewrite System. arXiv preprint arXiv:2412.01661.
memories. arXiv preprint arXiv:2212.10511.
Tang, J.; Zhang, Q.; Li, Y.; and Li, J. 2024. Grapharena:
Mavromatis, C.; and Karypis, G. 2024. GNN-RAG: Graph Benchmarking large language models on graph computa-
Neural Retrieval for Large Language Model Reasoning. tional problems. arXiv preprint arXiv:2407.00379.
arXiv preprint arXiv:2405.20139.
Tang, Y.; and Yang, Y. 2024. Multihop-rag: Benchmarking
Naeem, Z. A.; Ahmad, M. S.; Eltabakh, M.; Ouzzani, M.; retrieval-augmented generation for multi-hop queries. arXiv
and Tang, N. 2024. RetClean: Retrieval-Based Data Clean- preprint arXiv:2401.15391.
ing Using LLMs and Data Lakes. Proceedings of the VLDB
Endowment, 17(12): 4421–4424. Traag, V. A.; Waltman, L.; and Van Eck, N. J. 2019. From
Louvain to Leiden: guaranteeing well-connected communi-
Narayan, A.; Chami, I.; Orr, L.; and Ré, C. 2022. Can ties. Scientific reports, 9(1): 1–12.
Foundation Models Wrangle Your Data? Proceedings of the
Von Luxburg, U. 2007. A tutorial on spectral clustering.
VLDB Endowment, 16(4): 738–746.
Statistics and computing, 17: 395–416.
Nie, Y.; Kong, Y.; Dong, X.; Mulvey, J. M.; Poor, H. V.; Wang, J.; Fu, J.; Wang, R.; Song, L.; and Bian, J. 2025.
Wen, Q.; and Zohren, S. 2024. A Survey of Large Language PIKE-RAG: sPecIalized KnowledgE and Rationale Aug-
Models for Financial Applications: Progress, Prospects and mented Generation. arXiv preprint arXiv:2501.11551.
Challenges. arXiv preprint arXiv:2406.11903.
Wang, J.; Ning, H.; Peng, Y.; Wei, Q.; Tesfai, D.; Mao,
Nussbaum, Z.; Morris, J. X.; Duderstadt, B.; and Mulyar, A. W.; Zhu, T.; and Huang, R. 2024a. A Survey on Large
2024. Nomic Embed: Training a Reproducible Long Con- Language Models from General Purpose to Medical Appli-
text Text Embedder. arXiv:2402.01613. cations: Datasets, Methodologies, and Evaluations. arXiv
Peng, B.; Zhu, Y.; Liu, Y.; Bo, X.; Shi, H.; Hong, C.; Zhang, preprint arXiv:2406.10303.
Y.; and Tang, S. 2024. Graph retrieval-augmented genera- Wang, K.; Duan, F.; Wang, S.; Li, P.; Xian, Y.; Yin, C.; Rong,
tion: A survey. arXiv preprint arXiv:2408.08921. W.; and Xiong, Z. 2023. Knowledge-driven cot: Exploring
Qian, Y.; He, Y.; Zhu, R.; Huang, J.; Ma, Z.; Wang, H.; faithful reasoning in llms for knowledge-intensive question
Wang, Y.; Sun, X.; Lian, D.; Ding, B.; et al. 2024. UniDM: A answering. arXiv preprint arXiv:2308.13259.
Wang, S.; Xu, T.; Li, H.; Zhang, C.; Liang, J.; Tang, J.; graph construction and reasoning: Recent capabilities and
Yu, P. S.; and Wen, Q. 2024b. Large language mod- future opportunities. World Wide Web, 27(5): 58.
els for education: A survey and outlook. arXiv preprint
arXiv:2403.18105.
Wang, Y.; Lipka, N.; Rossi, R. A.; Siu, A.; Zhang, R.; and
Derr, T. 2024c. Knowledge graph prompting for multi-
document question answering. In Proceedings of the AAAI
Conference on Artificial Intelligence, volume 38, 19206–
19214.
Wu, J.; Zhu, J.; Qi, Y.; Chen, J.; Xu, M.; Menolascina, F.;
and Grau, V. 2024a. Medical graph rag: Towards safe medi-
cal large language model via graph retrieval-augmented gen-
eration. arXiv preprint arXiv:2408.04187.
Wu, S.; Xiong, Y.; Cui, Y.; Wu, H.; Chen, C.; Yuan, Y.;
Huang, L.; Liu, X.; Kuo, T.-W.; Guan, N.; et al. 2024b.
Retrieval-augmented generation for natural language pro-
cessing: A survey. arXiv preprint arXiv:2407.13193.
Xu, S.; Pang, L.; Yu, M.; Meng, F.; Shen, H.; Cheng, X.; and
Zhou, J. 2024. Unsupervised Information Refinement Train-
ing of Large Language Models for Retrieval-Augmented
Generation. arXiv preprint arXiv:2402.18150.
Xu, X.; Yuruk, N.; Feng, Z.; and Schweiger, T. A. 2007.
Scan: a structural clustering algorithm for networks. In Pro-
ceedings of the 13th ACM SIGKDD international conference
on Knowledge discovery and data mining, 824–833.
Yang, Z.; Qi, P.; Zhang, S.; Bengio, Y.; Cohen, W. W.;
Salakhutdinov, R.; and Manning, C. D. 2018. HotpotQA: A
dataset for diverse, explainable multi-hop question answer-
ing. arXiv preprint arXiv:1809.09600.
Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan,
K.; and Cao, Y. 2022. React: Synergizing reasoning and act-
ing in language models. arXiv preprint arXiv:2210.03629.
Yu, H.; Gan, A.; Zhang, K.; Tong, S.; Liu, Q.; and Liu, Z.
2024. Evaluation of Retrieval-Augmented Generation: A
Survey. arXiv preprint arXiv:2405.07437.
Zhang, N.; Choubey, P. K.; Fabbri, A.; Bernadett-Shapiro,
G.; Zhang, R.; Mitra, P.; Xiong, C.; and Wu, C.-S. 2024a.
SiReRAG: Indexing Similar and Related Information for
Multihop Reasoning. arXiv preprint arXiv:2412.06206.
Zhang, Q.; Hong, X.; Tang, J.; Chen, N.; Li, Y.; Li, W.;
Tang, J.; and Li, J. 2024b. Gcoder: Improving large lan-
guage model for generalized graph problem solving. arXiv
preprint arXiv:2410.19084.
Zhao, P.; Zhang, H.; Yu, Q.; Wang, Z.; Geng, Y.; Fu, F.;
Yang, L.; Zhang, W.; and Cui, B. 2024. Retrieval-augmented
generation for ai-generated content: A survey. arXiv preprint
arXiv:2402.19473.
Zheng, Y.; Gan, W.; Chen, Z.; Qi, Z.; Liang, Q.; and Yu,
P. S. 2024. Large language models for medicine: a survey.
International Journal of Machine Learning and Cybernet-
ics, 1–26.
Zhou, Y.; Cheng, H.; and Yu, J. X. 2009. Graph clustering
based on structural/attribute similarities. Proceedings of the
VLDB Endowment, 2(1): 718–729.
Zhu, Y.; Wang, X.; Chen, J.; Qiao, S.; Ou, Y.; Yao, Y.; Deng,
S.; Chen, H.; and Zhang, N. 2024. Llms for knowledge
 Method details of ArchRAG After all nodes at layer i (i < L) have been inserted, we
LLM-based hierarchical clustering check each node at layer i + 1 to ensure it has a inter-layer
link, confirming the traverse from the higher layer to the
For example, as shown in the second step of offline index- lower layer.
ing in Figure 2, LLM-based hierarchical clustering first en-
hances the original KG at layer L0 by adding similar edges.
Next, the weight of each edge is calculated based on the Algorithm 3: C-HNSW construction
strength of the relationship between the node embeddings, input : The Hierarchical community HC, KG G(V, E),
and a weighted clustering algorithm is applied to obtain four maximum number of connections for each node
communities. Based on the existence of links between nodes M.
within a community, the topology of communities at layer 1 H ← ∅;
L1 is constructed, resulting in a graph of communities. This 2 V ← {HC ∪ V } // Get all nodes of each
process is repeated, ultimately generating a clustering result layer.
consisting of three layers of hierarchical communities. 3 for each layer l ← L · · · 0 do
4 for each node v ∈ Vl do
5 if l ̸= L then
More details of C-HNSW 6 R ← SearchLayer(Gl = (Vl , El ), q, s, 1);
• Introduction of HNSW. We provide a brief introduc- 7 c ← get the nearest node from R;
tion to HNSW, an efficient Approximate Nearest Neighbor 8 s ← node in layer l − 1 via c’s inter-layer link;
Search (ANNS) technique for vector databases. 9 if s is null or d(c, s) > d(c, v) then
10 update v as c’s inter-layer link
Definition 1 (Hierarchical Navigable Small World (HNSW)
11 if l = L or s is null then s ← random node in
(Malkov and Yashunin 2018)). HNSW is a graph-based layer l;
ANNS algorithm that consists of a multi-layered index struc- 12 R ← SearchLayer(Gl = (Vl , El ), q, s, M );
ture, where each node uniquely corresponds to a vector in 13 add edges between v and R, update El ;
the database. Given a set S containing n vectors, the con- 14 H ← H ∪ Gl = (Vl , El )
structed HNSW can be represented as a pair H = (G, C). 15 return H;
G = {G0 , G1 , . . . , GL } is a set of simple graphs (also
called layers) Gi = (Vi , Ei ), where i ∈ {0, 1, . . . , L} and
VL ⊂ VL−1 ⊂ · · · ⊂ V1 ⊂ V0 = S. C records the inter-
layer mappings of edges between the same node across ad- Complexity analysis of ArchRAG
SL−1
jacent layers C = i=0 {(v, ϕ(v))|v ∈ Vi , ϕ(v) ∈ Vi+1 },
We now analyze the complexity of our ArchRAG approach.
where ϕ(v) : Vi → Vi+1 is the mapping function for the
Since the token cost is more important in the era of LLM,
same node across two adjacent layers.
we replace the space complexity with the token cost.
The nodes in the multi-layer graph of HNSW are orga- The offline indexing process of ArchRAG includes the
nized in a nested structure, where each node at each layer KG construction, hierarchical clustering, and C-HNSW con-
is connected to its nearest neighbors. During a query, the struction. The time complexity and token usage are as fol-
search begins at the top layer and quickly identifies the node lows:
closest to q through a greedy search. Then, through inter-
layer mapping, the search proceeds to the next lower layer. Lemma 1. Given a large text corpus or a large set of text
This process continues until all approximate nearest neigh- documents with a total of D tokens, the time complexity of
1−aL
bors are identified in G0 . the offline indexing process of ArchRAG is O(I D w + 1−a (n∗
• The construction of C-HNSW. A naive approach to t+ID w + π(m) + n log n)), where I is the generation time
build the C-HNSW index is to build nodes first and then of the LLM for a single inference, w is the specified token
establish the two types of links by finding the nearest neigh- size of one chunk, n and m are the number of entities and
bors of each node. However, the process of finding the near- relations in the extracted KG, L is the height of the resulting
est neighbor is costly. To accelerate the construction, we pro- hierarchical community structure, and a is the average ratio
pose a top-down approach by borrowing the idea of HNSW of the number of nodes between two consecutive layers, with
construction. The construction of C-HNSW is illustrated in 0 < a < 1. For a given embedding model, the computation
Algorithm 3. The construction algorithm of C-HNSW fol- time for the embedding of an entity description is denoted by
lows a top-down approach. Using the query process of C- t, while a specific clustering method is typically a function
HNSW, we obtain the M nearest neighbors of each node in of m, represented as π(m).
its layer, and the inter-layer links are continuously updated
during this process. When inserting a node x at layer i , if
the nearest neighbor at the layer i + 1 is cj , the inter-layer Proof. For the corpus D, we use the LLM to infer and ex-
link of cj is updated in the following two cases: tract the KG from each chunk of size w, resulting in a cost of
O(I Dw ) for constructing the KG. Since the size of all com-
• Node cj does not have a inter-layer link to the layer i. munity summaries in a single layer would not exceed the
• The distance from node cj to node x is smaller than the length of the corpus, their LLM inference time is also less
distance from cj to its previous nearest neighbor x′ , i.e., than O(I D w ). For each layer of clustering, the embedding of
d(cj , x) < d(cj , x′ ). each point must be computed, and clustering is performed
with time complexity of π(m). In the C-HNSW construc- In practice, multiple retrieved contents are combined, al-
tion, each point requires O(nlogn) time to perform the k- lowing the LLM to analyze them together, provided the total
nearest neighbor search and establish connections, which is token count does not exceed the token size limit. As a result,
similar to the proof in (Malkov and Yashunin 2018). For the time and token usage for online retrieval are lower than
an L-layer multi-layer graph structure, where the number of those required for analysis.
nodes decreases by a factor of a between two consecutive
layers, the increase in clustering and C-HNSW construction Experimental details
L
time is given by: 1−a
1−a . Metrics
Lemma 2. Given a large text corpus or a large set of This section provides additional details on the metrics.
text documents with a total of D tokens, the number of to- • Metrics for specific QA tasks. We choose accuracy
kens used in the offline indexing process of ArchRAG is as the evaluation metric based on whether the gold answers
O(D(1 + 1−a
L are included in the model’s generations rather than strictly
1−a )), where L is the height of the resulting hi- requiring an exact match, following (Schick et al. 2024;
erarchical community structure and a is the average ratio of
Mallen et al. 2022; Asai et al. 2023). This is because LLM
the number of nodes between two consecutive layers, with
outputs are typically uncontrollable, making it difficult for
0 < a < 1.
them to match the exact wording of standard answers. Sim-
ilarly, we choose recall as the metric instead of precision,
Proof. Based on the above proof, the token cost for con- as it better reflects the accuracy of the generated responses.
structing the KG is O(D), while the token cost for the sum- Additionally, when calculating recall, we adopt the same ap-
L
maries does not exceed O(D 1−a
1−a ). proach as previous methods (Gutiérrez et al. 2024; Asai et al.
2023): if the golden answer or the generated output contains
Generally, L is O(log n), where n is the number of ex- “yes” or “no”, the recall for that question is set to 0. There-
tracted entities. However, due to the constraints of commu- fore, the recall metric is not perfectly correlated with accu-
nity clustering, L is typically constant, usually no greater racy.
than 5. • Metrics for abstract QA tasks. Following exist-
Next, we analyze the time complexity and token usage in ing works, we use an LLM to generate abstract questions,
the online query process of the ArchRAG. with the prompts shown in Figure 13, defining ground truth
for abstract questions, particularly those involving complex
Lemma 3. Given a C-HNSW with L layers constructed from high-level semantics, poses significant challenges. We build
a large text corpus or a large set of text documents, the time on existing works (Edge et al. 2024; Guo et al. 2024) to
complexity of a sequentially executed single online retrieval address this and adopt an LLM-based multi-dimensional
query in ArchRAG is O(e + LkI + Lk log(n)), where I is comparison method (including comprehensiveness, diver-
the generation time of the LLM for a single inference, e is sity, empowerment, and overall). We employ a robust LLM,
the time cost of computing the query embedding, k is the specifically GPT-4o, to rank each baseline against our
number of nodes retrieved at each layer, and n is the number method. Figure 14 shows the evaluation prompt we use.
of nodes at the lowest layer in C-HNSW. • Metrics of community quality. We select the follow-
ing metrics to evaluate the quality of the community:
Proof. ArchRAG first computes the embedding of the
query, which takes O(e) time. For each layer, querying one 1. Calinski-Harabasz Index (CHI) (Caliński and Harabasz
nearest neighbor takes no more than O(log(n)) time, similar 1974): A higher value of CHI indicates better cluster-
to the proof in (Malkov and Yashunin 2018). In the Adap- ing results because it means that the data points are more
tive filtering-based generation, the content of each query is spread out between clusters than they are within clusters.
analyzed and inferred, requiring O(LkI) time. Therefore, It is an internal evaluation metric where the assessment
the total time for the online retrieval query is O(e + LkI + of the clustering quality is based solely on the dataset and
Lk log(n)). the clustering results and not on external ground-truth la-
bels. The CHI is calculated by between-cluster separation
Lemma 4. Given a C-HNSW with L layers constructed from and within-cluster dispersion:
a large text corpus or a large set of text documents, the PC 2
number of tokens used for a single online retrieval query N −C i=1 ni ||ci − c||
CHI = PC P . (3)
in ArchRAG is O(kL(c + P ))), where k is the number of C −1 i=1 x∈Ci ||x − ci ||
2
neighbors retrieved at each layer, c is the average token of
the retrieved content, and P is the token of the prompt. N is the number of nodes. C is the number of [Link]
is the number of nodes in cluster i. Ci is the i−th cluster.
Proof. The token consumption for analyzing all retrieved in- ci is the centroid of cluster Ci . c is the overall centroid of
formation is O(kL(c + P ))), while the token consumption the datasets. x is the feature of the target node.
for generating the final response is of constant order. There- 2. Cosine Similarity (Sim) (Charikar 2002): Cosine similar-
fore, the total token consumption for the online retrieval is ity is a measure of similarity between two non-zero vec-
O(kL(c + P ))). tors defined in an inner product space. In this paper, for
 each cluster, we calculate the similarity between the cen- We compare our hierarchical search based on C-HNSW
troid of this cluster and each node in this cluster: with a baseline approach (Base-HNSW), which indepen-
C dently builds a vector index for attributed communities at
1 XX
Sim = Cosine(x, ci ). (4) each layer and performs retrieval separately for each. As
N i=1 shown in Figure 5, on the large-scale synthetic dataset, C-
x∈Ci
HNSW achieves up to a 5.4× speedup (On level 1, C-HNSW
N is the number of nodes. C is the number of clusters.
takes 1.861 seconds, while Base-HNSW takes 10.125 sec-
Ci is the i-th cluster. ci is the centroid of cluster Ci . x is
onds.) and is on average 3.5× faster than Base-HNSW. Ad-
the feature of the target node.
Pn ditional details are provided in the appendix.
xi yi
Cosine(x, y) = pPn i=1 2
pPn
2
. (5)
i=1 xi i=1 yi
Base-HNSW C-HNSW
15
Implementation details

time (s)
10
We implement our ArchRAG in Python, while C-HNSW
5
is implemented in C++ and provides a Python interface
for integration. We implement C-HNSW using the FAISS 0
0 1 2 3 4 5 6 7 8 9 10 Avg.
framework and employ the inner product metric to mea- Level
sure the proximity between two vectors. All the experi-
ments were conducted on a Linux operating system run- Figure 5: C-HNSW and Base-HNSW query efficiency.
ning on a machine with an Intel Xeon 2.0 GHz CPU,
1024GB of memory, and 8 NVIDIA GeForce RTX A5000 We also conducted experiments on a synthetic dataset of
GPUs, each with 24 GB of memory. All methods uti- 1024-dimensional vectors, keeping all other parameters un-
lize 10 concurrent LLM calls, and to maintain consis- changed. The results are shown in Figure 6. As 1024 di-
tency, other parallel computations in the method, such as mensions are more representative of commonly used high-
embedding calculations, also use 10 concurrent threads. dimensional embeddings, our method still achieves over 5×
Figure 15 demonstrates the prompt used in Adaptive speedup in the best case and an average speedup of 3× com-
filtering-based generation. Please refer to our repository pared to the baseline.
([Link] to
view the detailed prompts. Base-HNSW C-HNSW
• Details of clustering methods. The graph augmen-
tation methods we choose are the KNN algorithm, which 15
time (s)

computes the similarity between each node, and CODI- 10

CIL (Ruan, Fuhry, and Parthasarathy 2013), which selects 5
and adds the top similar edges to generate better clustering 0
0 1 2 3 4 5 6 7 8 9 10 Avg.
results. In the KNN method, we set the K value as the aver-
Level
age degree of nodes in the KG.
Figure 6: C-HNSW and Base-HNSW query efficiency on
Additional experiments 1024-dimensional vector dataset.
• Efficiency of hierarchical search (C-HNSW). To evalu-
ate the efficiency of C-HNSW, we conduct experiments on a • Efficiency of indexing phrase. Figure 7 shows the in-
synthetic hierarchical dataset comprising 11 layers. The bot- dex construction time and token usage for different methods.
tom layer (Layer 0) contains 10 million nodes, and the num- The cost of building an index for ArchRAG is similar to that
ber of nodes decreases progressively across higher layers by of GraphRAG, but due to the need for community summa-
randomly dividing each layer’s size by 3 or 4. The top layer rization, both take a higher time cost and token usage than
(Layer 10), for example, contains only 74 nodes. This hier- HippoRAG.
archical structure simulates the process of LLM-based hier-
archical clustering. Each node is assigned a randomly gener-
RAPTOR HippoRAG LightRAG
ated 3072-dimensional vector, simulating high-dimensional
embeddings such as those produced by text or image en- GraphRAG ArchRAG
coders (e.g., text-embedding-3-large, used in Chat- OOT 100
token (M)

GPT, can generate 3072-dimensional vectors, and text-

time (s)

embedding-v3 can generate 1024-dimensional vectors). For 104

OOT

10
each layer, we generate 200 random queries and compute
the top-5 nearest neighbors for each query. Both C-HNSW 103Multihop-RAGHotpotQA 1Multihop-RAGHotpotQA
and Base-HNSW are configured with identical parameters: (a) Time cost (b) Token cost
M = 32, ef Search = 100, and ef Construction = 100.
Importantly, our method maintains comparable retrieval ac-
curacy to Base-HNSW, with recall of 0.5537 and 0.6058, Figure 7: Comparison of indexing efficiency.
respectively.
 Table 5: Comparing ArchRAG with other RAG methods on the specific QA tasks under different LLM backbone models.

Multihop-RAG HotpotQA
LLM backbone Methods
(Accuracy) (Recall) (Accuracy) (Recall)
Vanilla RAG 58.6 31.4 50.6 56.1
HippoRAG 38.9 19.1 51.3 56.8
Llama3.1-8B
RAPTOR 59.1 34.1 N/A N/A
ArchRAG 68.8 37.2 65.4 69.2
Vanilla RAG 65.9 32.8 60.7 65.9
HippoRAG 68.9 31.4 58.0 62.3
GPT-3.5-turbo
RAPTOR 64.4 34.6 N/A N/A
ArchRAG 67.2 31.5 62.8 65.0
Vanilla RAG 71.4 32.8 68.2 70.1
HippoRAG 70.5 31.6 65.0 68.5
GPT-4o-mini
RAPTOR 70.1 32.6 N/A N/A
ArchRAG 77.3 33.8 69.9 73.8

To further demonstrate the effectiveness of ArchRAG, we KNN CODICIL

conduct the following experiments:
• Effectiveness of LLM backbones. Given the limited 4 4

CHI

CHI
budget, we restrict our evaluation to GPT-4o-mini and GPT-
2 2
3.5-turbo as the LLM backbones, and compare a representa-
tive subset of strong RAG methods on the HotpotQA and 0 Leiden Spectral SCAN Node2Vec
0 Leiden Spectral SCAN Node2Vec

Multihop-RAG datasets. As strong LLMs with hundreds (a) Multihop-RAG (b) HotpotQA
of billions of parameters (e.g., GPT-3.5-turbo) possess en-
hanced capabilities, our proposed ArchRAG may also ben- Figure 8: Community quality evaluated by CH Index.
efit from performance improvement. As shown in Table 5,
the results of Llama 3.1-8B are similar to those of GPT-3.5-
turbo, as Llama3.1’s capabilities are comparable to those of KNN CODICIL
GPT-3.5-turbo (Dubey et al. 2024). GPT-4o-mini performs
better than other LLM backbones because of its exceptional 1 1
0.8
Sim

Sim
reasoning capabilities. 0.8
Besides, we have compared several strong RAG baselines 0.6
under different LLM backbones. As LLMs’ parameters and 0.4 Leiden Spectral SCAN Node2Vec
0.6 Leiden Spectral SCAN Node2Vec
reasoning capabilities increase, all RAG approaches benefit
(a) Multihop-RAG (b) HotpotQA
from performance gains, especially HippoRAG. ArchRAG
consistently achieves state-of-the-art performance across
most settings. Figure 9: Community quality evaluated by Cosine Similar-
• Community quality of different clustering meth- ity.
ods. We evaluate the community quality of our proposed
LLM-based hierarchical clustering framework using four
clustering algorithms (weighted Leiden, Spectral Cluster- clustering algorithm, we evaluate the quality of communities
ing (Von Luxburg 2007), SCAN (Xu et al. 2007), and (i.e., CHI and Cosine Similarity) generated by our attributed
node2vec (Grover and Leskovec 2016) with KMeans), com- clustering algorithm compared to those produced by the Lei-
bined with two graph augmentation techniques (the KNN den algorithm, which is used in GraphRAG for structural
algorithm and CODICIL (Ruan, Fuhry, and Parthasarathy clustering. As shown in Table 7, our attribute-based cluster-
2013)). The resulting communities are assessed using the ing consistently yields higher-quality communities.
Calinski-Harabasz Index (CHI) (Caliński and Harabasz • More experiments on the additional dataset. We
1974) and Cosine Similarity (Sim) (Charikar 2002), where further conduct experiments on the RAG-QA Arena
higher values indicate better quality. Further details on the dataset (Han et al. 2024), a high-quality, multi-domain
clustering implementation and evaluation metrics can be benchmark featuring human-annotated, coherent long-form
found in the appendix. Figures 8 and 9 show that combin- answers. To the best of our capability, we use publicly avail-
ing KNN or CODICIL with the weighted Leiden algorithm able data from five domains (including lifestyle, recreation,
significantly enhances community detection quality. science, technology, and writing), selecting 200 questions
• Effectiveness of our attributed clustering algorithm. per domain. Following prior work, we employ LLMs as
To further demonstrate the effectiveness of our attributed evaluators to compare the RAG-generated responses with
Table 6: Comparing ArchRAG with other RAG methods on the RAG-QA Area dataset. Each entry denotes the win ratio and
win + tie ratio of the corresponding method against the ground-truth annotations, based on LLM evaluation.

Method Lifestyle Recreation Science Technology Writing

Vanilla RAG 17.5 / 20.5 17.0 / 25.0 32.5 / 37.0 28.5 / 34.0 15.0 / 16.5
HippoRAG 26.5 / 26.5 29.5 / 30.5 49.5 / 49.5 42.0 / 42.5 21.0 / 21.5
RAPTOR 17.0 / 19.5 18.5 / 24.5 39.0 / 45.0 33.0 / 35.5 25.0 / 26.5
ArchRAG 49.5 / 50.0 41.5 / 41.5 56.0 / 56.0 59.0 / 59.5 45.0 / 45.0

Table 7: Comparison of Community Quality between Our Table 8: Distribution of HippoRAG’s ER Errors
Attributed Clustering Method and Leiden
Low-Quality
Datasets Null Entity Rate
Multihop-RAG HotpotQA Entity Rate
Method
(CHI) (Sim) (CHI) (Sim) Multihop-RAG 1.3% 11.9%
Leiden 3.02 0.71 3.42 0.71 HotpotQA 5.0% 15.8%
Ours 4.68 0.89 4.82 0.88

performs worse than retrieval-only methods while outper-

ground-truth answers in terms of win ratio and win + tie forming retrieval-based methods on the HotpotQA dataset.
ratio. We evaluate the top-performing methods (including This is mainly because, on the HotpotQA dataset, passages
Vanilla RAG, HippoRAG, RAPTOR, and our ArchRAG) are segmented by the expert annotators; that is, passages can
and present the results in terms of the win ratio and win + provide more concise information, whereas on Multihop-
tie ratio against the ground-truth annotations in the table be- RAG, passages are segmented based on chunk size, which
low. As shown in Table 6, ArchRAG consistently achieves may cause the LLM to lose context and produce incorrect
state-of-the-art performance across all evaluated settings. answers. Besides, HippoRAG also suffers from inaccurate
entity recognition. In addition, we also provide a brief anal-
Accuracy Recall ysis of why some graph-based RAG methods underperform.
80 80 Specifically, they often fail to retrieve relevant information
Metrics

Metrics

during the search process. For example:

60 60
40 40 • GraphRAG: The suboptimal performance of GraphRAG
20 20 has been widely observed in research. For instance,
2 3 5 7 9 2 3 5 7 9 Zhang et al. (Zhang et al. 2024a) find that GraphRAG
(a) Multihop-RAG (b) HotpotQA tends to respond with ”I don’t know” when retrieving ir-
relevant content, indicating that it prefers not to give a
Figure 10: Comparative analysis of the different numbers of concrete answer. PIKE-RAG (Wang et al. 2025) identi-
retrieval elements in ArchRAG. fies that GraphRAG tends to echo the query and include
meta-information about answers within its graph struc-
• Effect of k values. We compare the performance of ture. Through manual verification, we confirm the pres-
ArchRAG under different retrieved elements. As shown in ence of similar issues in our experimental results. In par-
Figure 10, the performance of ArchRAG shows little vari- ticular, GraphRAG-Local Search also frequently returns
ation when selecting different retrieval elements (i.e., com- irrelevant content.
munities and entities in each layer). This suggests that the • HippoRAG/LightRAG: We attribute part of HippoRAG’s
adaptive filtering process can reliably extract the most rele- suboptimal performance to entity recognition issues.
vant information from the retrieval elements and integrate it HippoRAG first extracts relevant entities from questions
to generate the answer. and then retrieves chunks based on these entities. How-
• Case study. We present an additional case study from ever, it sometimes fails to identify appropriate entities
Multihop-RAG. As shown in Figure 11, only our method (Entity Recognition Errors, ER Errors), and we conduct
generates the correct answer, while others either provide in- corresponding experiments, as shown in Table 8. For ex-
correct or irrelevant information. We only show the core ample, on HotpotQA, HippoRAG fails to detect entities
output for brevity, with the remaining marked as “¡/¿”. in 5.0% of questions and identifies low-quality ones in
We also show the retrieval and adaptive filtering process 15.8% (e.g., less than 3 characters, numeric-only terms,
of ArchRAG in Figure 12. The results demonstrate that or sentence entities with more than 5 words). These is-
ArchRAG effectively retrieves relevant information and fil- sues make it difficult to match accurate entities in the
ters out noise, leading to a correct final answer. KG, resulting in irrelevant retrieval and suboptimal per-
• Discussion of the performance of other graph-based formance. Similar issues also impair LightRAG’s perfor-
RAG methods. On the Multihop-RAG dataset, HippoRAG mance because it requires accurate topic identification
from questions. We also observe that as the LLM’s ca-
pability improves, the performance of these methods in-
creases significantly, as shown in the experiment Effec-
tiveness of LLM backbone.
 Question & Ground-truth

Question: Who is the individual associated with generative AI technology that was reportedly ousted from a leading AI
organization, but is recognized for brilliance and generosity in the industry, and is currently planning to launch a new
venture according to reports from TechCrunch and Fortune?
Ground-truth: Sam Altman

Responses of Different Methods

Zero-shot
Jesse Schell.
He is the CTO of Calm and was the lead designer on the “Star Wars” theme park ride at Disney.

Vanilla RAG
It seems like there are multiple articles and topics discussed in this text. Here’s a brief summary of each section:
[Link] predictions. The article discusses the future of robotics, with predictions made by Brian Heater. · · ·
[Link]’s Week in Review. This section summarizes various tech industry happenings. · · ·
[Link] gets a new CEO. Bumble announces a change in leadership. · · ·

HippoRAG
There is no question in the provided text. It appears to be a collection of news articles and updates related to OpenAI’s
ChatGPT. · · ·

GraphRAG-Global
Key Points and Implications
The individual associated with generative AI technology who was reportedly ousted is Andrew Ng. · · ·
According to multiple analysts, Andrew Ng [Data: Reports (5, 6)] · · ·
Implications of the New Venture · · ·

ArchRAG
Sam Altman.
This information is supported by multiple analysts’ reports, which highlight Sam Altman’s involvement in various AI-
related projects. The direct answer is simply “Sam Altman”. · · ·
Note: I have not included any information from Analyst 2, 3, or 7 as their reports are not relevant. · · ·

Figure 11: Case study of responses by different RAG methods on a question from the Multihop-RAG dataset. RED TEXT
indicates incorrect content, CYAN TEXT denotes correct content, and · · · marks omitted irrelevant parts.

Top Retrieved Entities:

- Entity 1058: OPENAI
Description: OpenAI is a tech company founded by Sam Altman ...

- Community 4097: Microsoft, OpenAI, and AI Regulation

Summary: Discussion between Satya Nadella and Sam Altman ...

Adaptive Filtering Result:

- Sam Altman is among the backers of an AI startup.
- Score: 80.0 (Reports: 1) ...

Figure 12: ArchRAG Retrieval & Filtering Output. ... marks omitted irrelevant parts.
Prompt for generating abstract questions
Prompt:
Given the following description of a dataset:
{description}
Please identify 5 potential users who would engage with this dataset. For each user, list 5 tasks they would
perform with this dataset. Then, for each (user, task) combination, generate 5 questions that require a high-level
understanding of the entire dataset.
Output the results in the following structure:
- User 1: [user description]
- Task 1: [task description]
- Question 1:
- Question 2:
- Question 3:
- Question 4:
- Question 5:
- Task 2: [task description]
...
- Task 5: [task description]
- User 2: [user description]
...
- User 5: [user description]
...
Note that there are 5 users and 5 tasks for each user, resulting in 25 tasks in total. Each task should have 5
questions, resulting in 125 questions in total. The Output should present the whole tasks and questions for each
user.
Output:

Figure 13: The prompt for generating abstract questions.

Prompt for LLM-based multi-dimensional comparison
Prompt:
You will evaluate two answers to the same question based on three criteria: Comprehensiveness, Diversity,
Empowerment, and Directness.
• Comprehensiveness: How much detail does the answer provide to cover all aspects and details of the ques-
tion?
• Diversity: How varied and rich is the answer in providing different perspectives and insights on the question?
• Empowerment: How well does the answer help the reader understand and make informed judgments about
the topic?
• Directness: How specifically and clearly does the answer address the question?
For each criterion, choose the better answer (either Answer 1 or Answer 2) and explain why. Then, select an
overall winner based on these four categories.
Here is the question:
Question: {query}
Here are the two answers:
Answer 1: {answer1}
Answer 2: {answer2}
Evaluate both answers using the four criteria listed above and provide detailed explanations for each criterion.
Output your evaluation in the following JSON format:
{
"Comprehensiveness": {
"Winner": "[Answer 1 or Answer 2]",
"Explanation": "[Provide one sentence explanation here]"
},
"Diversity": {
"Winner": "[Answer 1 or Answer 2]",
"Explanation": "[Provide one sentence explanation here]"
},
"Empowerment": {
"Winner": "[Answer 1 or Answer 2]",
"Explanation": "[Provide one sentence explanation here]"
},
"Overall Winner": {
"Winner": "[Answer 1 or Answer 2]",
"Explanation": "[Briefly summarize why this answer is the overall
winner]"
}
}
Output:

Figure 14: The prompt for the evaluation of abstract QA.

Prompt for Adaptive filtering-based generation
Filter Prompt:
# Role
You are a helpful assistant responding to questions about data in the tables provided.
# Goal
Generate a response consisting of a list of key points that respond to the user’s question, summarizing all
relevant information in the input data tables. You should use the data provided in the data tables below as the
primary context for generating the response. If you don’t know the answer or if the input data tables do not
contain sufficient information to provide an answer, just say so. Do not make anything up.
Each key point in the response should have the following element:
• Description: A comprehensive description of the point.
• Importance Score: An integer score between 0-100 that indicates how important the point is in answering
the user’s question. An ‘I don’t know’ type of response should have a score of 0.
The response should be JSON formatted as follows:
{"points": [
{
"description": "Description of point 1",
"score": score_value
},
// ... more points]}
# User Question
{user_query}
# Data tables
{context_data}
Output:

Merge Prompt:
# Role
You are a helpful assistant responding to questions and may use the provided data as a reference.
# Goal
You should incorporate insights from all the reports from multiple analysts who focused on different parts of
the dataset to support your answer. Please note that the provided information may contain inaccuracies or be
unrelated. If the provided information does not address the question, please respond using what you know:
• A response that utilizes the provided information, ensuring that all irrelevant details from the analysts’
reports are removed.
• A response to the user’s query based on your existing knowledge when ¡Analyst Reports¿ is empty.
The final response should merge the relevant information into a comprehensive answer that clearly explains
all key points and implications, tailored to the appropriate response length and format. Note that the analysts’
reports provided below are ranked in the descending order of importance. Do not include information where
the supporting evidence for it is not provided.
# Target response length and format
{response_format}
# User Question
{user_query}
# Analyst Reports
{report_data}
Output:

Figure 15: The prompt for adaptive filtering-based generation.